2024-09-17T17:17:15.154 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T17:17:15.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T17:17:15.262 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156 branch: wip-bharath10-testing-2024-09-17-1348 description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-hybrid} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: skanta@redhat.com first_in_suite: false job_id: '7909156' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath10-testing-2024-09-17-1348 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: false bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath10-testing-2024-09-17-1348 sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 3421 sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath10-testing-2024-09-17-1348 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXIDGWWnEt9ojuP6e23K4bQQwPKkBm2z5+4ShXYpsiSSjlgI5YVqM2pBwJ26fcJftxvQp8hL0gAiNDfjpexmJ8= smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFeZUbjSYefjq3WII5LE+SUxbHhqvlMBj2hyAD3HjQCVfS3IsozictrBx0nZ/UVzvcA7VtGWHeHQ+d7vYtTLeE= smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBUyqqqCxlxFiiUg4s+MWqz5xKdSLMrLuiohJ+HD9cZcpSNmVPvn9aS5oPSfMXrlBDErbS037L4Tlnw2L5LOpQg= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCp51lPnWmMwV1Js9tykoeu4DQ8nr+s+UJZqCq4ie4VrzHfSAFva3YFG0EsLBMq+y0gEIojWyVXuleVlWjUPiWo= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_15:01:33 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T17:17:15.262 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa; will attempt to use it 2024-09-17T17:17:15.263 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa/tasks 2024-09-17T17:17:15.263 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T17:17:15.265 INFO:teuthology.task.internal:Checking packages... 2024-09-17T17:17:15.286 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1889a165b5128787ec693450a3d1c5df5d6dc203' 2024-09-17T17:17:15.286 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T17:17:15.286 INFO:teuthology.packaging:ref: None 2024-09-17T17:17:15.286 INFO:teuthology.packaging:tag: None 2024-09-17T17:17:15.286 INFO:teuthology.packaging:branch: wip-bharath10-testing-2024-09-17-1348 2024-09-17T17:17:15.286 INFO:teuthology.packaging:sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:17:15.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath10-testing-2024-09-17-1348 2024-09-17T17:17:15.430 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4975.g1889a165 2024-09-17T17:17:15.432 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T17:17:15.438 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T17:17:15.438 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T17:17:15.475 INFO:teuthology.task.internal:Saving configuration 2024-09-17T17:17:15.491 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T17:17:15.500 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T17:17:15.524 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 17:11:12.439013', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXIDGWWnEt9ojuP6e23K4bQQwPKkBm2z5+4ShXYpsiSSjlgI5YVqM2pBwJ26fcJftxvQp8hL0gAiNDfjpexmJ8='} 2024-09-17T17:17:15.542 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 17:11:12.440486', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFeZUbjSYefjq3WII5LE+SUxbHhqvlMBj2hyAD3HjQCVfS3IsozictrBx0nZ/UVzvcA7VtGWHeHQ+d7vYtTLeE='} 2024-09-17T17:17:15.561 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 17:11:12.437365', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBUyqqqCxlxFiiUg4s+MWqz5xKdSLMrLuiohJ+HD9cZcpSNmVPvn9aS5oPSfMXrlBDErbS037L4Tlnw2L5LOpQg='} 2024-09-17T17:17:15.580 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 17:11:12.435456', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCp51lPnWmMwV1Js9tykoeu4DQ8nr+s+UJZqCq4ie4VrzHfSAFva3YFG0EsLBMq+y0gEIojWyVXuleVlWjUPiWo='} 2024-09-17T17:17:15.581 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T17:17:15.587 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-09-17T17:17:15.587 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-09-17T17:17:15.587 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-09-17T17:17:15.587 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-09-17T17:17:15.587 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T17:17:15.709 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f727a15bbe0>, signals=[15]) 2024-09-17T17:17:15.709 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T17:17:15.715 INFO:teuthology.task.internal:Opening connections... 2024-09-17T17:17:15.716 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:17:15.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:17:15.791 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:17:15.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:17:15.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:17:15.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:17:15.947 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:17:15.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:17:16.022 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T17:17:16.029 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-17T17:17:16.048 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-17T17:17:16.048 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-17T17:17:16.105 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T17:17:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T17:17:16.107 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-17T17:17:16.134 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-09-17T17:17:16.153 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-09-17T17:17:16.153 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-09-17T17:17:16.210 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-09-17T17:17:16.210 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T17:17:16.211 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T17:17:16.212 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-09-17T17:17:16.234 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-17T17:17:16.253 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-17T17:17:16.254 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-09-17T17:17:16.312 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T17:17:16.313 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T17:17:16.313 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-17T17:17:16.334 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-17T17:17:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-17T17:17:16.352 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-17T17:17:16.410 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T17:17:16.411 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T17:17:16.412 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-17T17:17:16.434 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T17:17:16.443 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T17:17:16.483 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T17:17:16.483 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-17T17:17:16.486 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-09-17T17:17:16.488 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-17T17:17:16.490 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-17T17:17:16.507 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T17:17:16.513 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T17:17:16.514 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-17T17:17:16.543 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-09-17T17:17:16.545 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-17T17:17:16.548 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-17T17:17:16.735 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T17:17:16.829 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T17:17:16.843 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T17:17:16.843 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T17:17:16.844 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T17:17:16.844 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T17:17:16.844 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T17:17:16.845 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T17:17:16.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:17:16.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:17:16.863 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-09-17T17:17:16.863 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-17T17:17:16.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:17:16.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:17:16.865 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-17T17:17:16.865 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-17T17:17:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-503.el9.x86_64 2024-09-17T17:17:16.921 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-503.el9.x86_64 2024-09-17T17:17:16.921 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-09-17T17:17:16.922 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-503.el9.x86_64 2024-09-17T17:17:16.922 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-503.el9.x86_64 2024-09-17T17:17:16.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-09-17T17:17:16.923 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-17T17:17:16.923 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-503.el9.x86_64 2024-09-17T17:17:16.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-09-17T17:17:16.924 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-503.el9.x86_64 2024-09-17T17:17:16.925 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-503.el9.x86_64 2024-09-17T17:17:16.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-09-17T17:17:19.739 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.3 MB 00:01 2024-09-17T17:17:21.491 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-17T17:17:21.807 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.3 MB 00:03 2024-09-17T17:17:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.3 MB 00:03 2024-09-17T17:17:23.499 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-17T17:17:23.892 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-17T17:17:25.337 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-17T17:17:25.867 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-17T17:17:27.883 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T17:17:29.384 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-17T17:17:29.741 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-17T17:17:30.305 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-17T17:17:31.233 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-17T17:17:31.251 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-17T17:17:31.263 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-17T17:17:32.037 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-17T17:17:32.693 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-17T17:17:32.780 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-17T17:17:33.735 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-17T17:17:33.856 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-17T17:17:35.902 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T17:17:37.635 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-17T17:17:37.884 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T17:17:37.884 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T17:17:37.984 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:17:37.989 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout:Install 4 Packages 2024-09-17T17:17:37.990 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:17:37.991 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 79 M 2024-09-17T17:17:37.991 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 126 M 2024-09-17T17:17:37.991 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:17:38.229 INFO:teuthology.orchestra.run.smithi026.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 344 kB/s | 36 kB 00:00 2024-09-17T17:17:38.604 INFO:teuthology.orchestra.run.smithi026.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-17T17:17:38.987 INFO:teuthology.orchestra.run.smithi026.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-17T17:17:39.066 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-17T17:17:39.296 INFO:teuthology.orchestra.run.smithi026.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-17T17:17:39.296 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:17:39.296 INFO:teuthology.orchestra.run.smithi026.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-17T17:17:39.420 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T17:17:39.724 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T17:17:39.725 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T17:17:39.824 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:17:39.830 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:17:39.831 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:17:39.832 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-09-17T17:17:39.832 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:17:39.832 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 79 M 2024-09-17T17:17:39.832 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 126 M 2024-09-17T17:17:39.832 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:17:40.005 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 139 kB/s | 36 kB 00:00 2024-09-17T17:17:40.367 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:17:40.368 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:17:40.964 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T17:17:40.964 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T17:17:41.066 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:17:41.072 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:17:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-17T17:17:41.072 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:17:41.072 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-17T17:17:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:17:41.074 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 79 M 2024-09-17T17:17:41.074 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 126 M 2024-09-17T17:17:41.074 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:17:41.200 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 341 kB/s | 36 kB 00:00 2024-09-17T17:17:41.351 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:17:41.351 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:17:41.437 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T17:17:41.437 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T17:17:41.532 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:17:41.538 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-17T17:17:41.539 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:17:41.540 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 79 M 2024-09-17T17:17:41.540 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 126 M 2024-09-17T17:17:41.540 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:17:41.576 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-17T17:17:41.673 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 409 kB/s | 36 kB 00:00 2024-09-17T17:17:42.009 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-17T17:17:42.065 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-17T17:17:42.285 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-17T17:17:42.285 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:17:42.285 INFO:teuthology.orchestra.run.smithi102.stdout:Total 65 MB/s | 79 MB 00:01 2024-09-17T17:17:42.616 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 34 MB/s | 35 MB 00:01 2024-09-17T17:17:42.970 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:17:43.166 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-17T17:17:43.166 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:17:43.166 INFO:teuthology.orchestra.run.smithi007.stdout:Total 48 MB/s | 79 MB 00:01 2024-09-17T17:17:43.335 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:17:43.335 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:17:43.883 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:17:43.912 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:17:44.254 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:17:44.254 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:17:44.285 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:17:44.285 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:17:44.549 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:17:44.629 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:45.078 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.8 MB/s | 29 MB 00:04 2024-09-17T17:17:45.254 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:17:45.254 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:17:45.845 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.6 MB/s | 16 MB 00:05 2024-09-17T17:17:46.718 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:17:47.162 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 4.7 MB/s | 35 MB 00:07 2024-09-17T17:17:47.162 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:17:47.162 INFO:teuthology.orchestra.run.smithi027.stdout:Total 11 MB/s | 79 MB 00:07 2024-09-17T17:17:47.356 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:17:47.378 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:47.444 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:47.506 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:17:47.876 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:17:48.282 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:17:48.282 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:17:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:17:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:49.328 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:17:49.328 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:17:50.184 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:50.315 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:51.314 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:51.450 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:51.907 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:17:52.593 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:17:52.633 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:17:52.681 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:55.143 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:55.224 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:17:55.542 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:17:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:56.583 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:17:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:17:57.940 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:00.082 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:18:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:01.764 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:06.166 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:36.837 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:37.228 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:37.228 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:18:37.228 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:18:37.228 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:18:38.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:38.064 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:18:38.064 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:18:38.064 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:38.064 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:38.065 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T17:18:38.065 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:38.065 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:18:38.065 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:18:38.393 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:38.477 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:38.516 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:38.516 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:38.517 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:38.518 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T17:18:38.518 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T17:18:38.518 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T17:18:38.518 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T17:18:38.518 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T17:18:38.518 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:38.576 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:38.577 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:38.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-09-17T17:18:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-09-17T17:18:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-09-17T17:18:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-09-17T17:18:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 82.el9 2024-09-17T17:18:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-09-17T17:18:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8274670 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T17:18:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:18:38.617 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T17:18:38.618 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T17:18:38.618 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 5.14.0-508.el9.x86_64 2024-09-17T17:18:38.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T17:18:38.914 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:38.914 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:18:38.914 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:18:38.914 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:18:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T17:18:39.747 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:39.748 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:18:39.748 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:18:40.112 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:40.187 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:40.187 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:40.187 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T17:18:40.187 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T17:18:40.187 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T17:18:40.187 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-17T17:18:40.188 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T17:18:40.188 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:40.246 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:40.246 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:40.246 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-09-17T17:18:40.278 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 82.el9 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8274670 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T17:18:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-09-17T17:18:40.280 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:18:40.281 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T17:18:40.282 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T17:18:40.282 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 5.14.0-508.el9.x86_64 2024-09-17T17:18:40.282 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T17:18:40.538 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:40.580 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:40.634 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T17:18:40.662 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-09-17T17:18:40.664 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-17T17:18:40.679 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.huyTAsQqDI 2024-09-17T17:18:40.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.huyTAsQqDI 2024-09-17T17:18:40.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.huyTAsQqDI 2024-09-17T17:18:40.892 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.huyTAsQqDI is 6KB 2024-09-17T17:18:40.930 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-09-17T17:18:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:18:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:18:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:18:40.940 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.huyTAsQqDI 2024-09-17T17:18:40.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T17:18:41.017 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T17:18:41.017 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T17:18:41.017 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T17:18:41.017 INFO:teuthology.orchestra.run.smithi026.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T17:18:41.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T17:18:41.183 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-09-17T17:18:41.735 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:41.735 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:18:41.735 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:18:41.735 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:41.735 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:41.735 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T17:18:41.736 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:41.736 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:18:41.736 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:18:42.069 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:42.161 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:42.161 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:42.161 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T17:18:42.161 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T17:18:42.162 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T17:18:42.162 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T17:18:42.162 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T17:18:42.162 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:42.216 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T17:18:42.220 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:42.220 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:42.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-17T17:18:42.244 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-09-17T17:18:42.251 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-17T17:18:42.257 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-17T17:18:42.257 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-17T17:18:42.257 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-17T17:18:42.257 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-17T17:18:42.258 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-17T17:18:42.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:18:42.260 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T17:18:42.260 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T17:18:42.260 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T17:18:42.260 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-17T17:18:42.260 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:18:42.260 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T17:18:42.261 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T17:18:42.261 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-508.el9.x86_64 2024-09-17T17:18:42.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T17:18:42.267 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.TvFakQt6gb 2024-09-17T17:18:42.267 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TvFakQt6gb 2024-09-17T17:18:42.340 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.TvFakQt6gb 2024-09-17T17:18:42.482 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.TvFakQt6gb is 6KB 2024-09-17T17:18:42.531 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.TvFakQt6gb 2024-09-17T17:18:42.546 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T17:18:42.609 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T17:18:42.609 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T17:18:42.609 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T17:18:42.609 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T17:18:42.611 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T17:18:42.767 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-09-17T17:18:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-17T17:18:44.220 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T17:18:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-17T17:18:44.242 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-17T17:18:44.257 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.Zw761weQhR 2024-09-17T17:18:44.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Zw761weQhR 2024-09-17T17:18:44.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.Zw761weQhR 2024-09-17T17:18:44.462 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.Zw761weQhR is 6KB 2024-09-17T17:18:44.511 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.Zw761weQhR 2024-09-17T17:18:44.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T17:18:44.587 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T17:18:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T17:18:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T17:18:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T17:18:44.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T17:18:44.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-17T17:18:46.306 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:46.348 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T17:18:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T17:18:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T17:18:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:18:47.486 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:18:47.864 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:47.997 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:47.997 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:47.998 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T17:18:47.998 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T17:18:47.998 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T17:18:47.998 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-17T17:18:47.998 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T17:18:47.998 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T17:18:48.057 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T17:18:48.058 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T17:18:48.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-09-17T17:18:48.101 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 82.el9 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8274670 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T17:18:48.102 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:18:48.103 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T17:18:48.104 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T17:18:48.104 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-508.el9.x86_64 2024-09-17T17:18:48.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T17:18:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-17T17:18:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T17:18:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-17T17:18:50.217 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-17T17:18:50.233 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.X1pJjsiRoV 2024-09-17T17:18:50.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.X1pJjsiRoV 2024-09-17T17:18:50.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.X1pJjsiRoV 2024-09-17T17:18:50.447 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.X1pJjsiRoV is 6KB 2024-09-17T17:18:50.497 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.X1pJjsiRoV 2024-09-17T17:18:50.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T17:18:50.577 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T17:18:50.577 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T17:18:50.577 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T17:18:50.577 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T17:18:50.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T17:18:50.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-17T17:19:11.206 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T17:19:11.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-17T17:19:11.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:12.771 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-17T17:19:12.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-17T17:19:12.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:14.756 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T17:19:14.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T17:19:14.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:20.750 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-17T17:19:20.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:19:20.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:29.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-09-17T17:19:31.137 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-17T17:19:33.185 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-17T17:19:38.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-17T17:19:38.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:39.069 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-17T17:19:40.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-17T17:19:40.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:42.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T17:19:42.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:43.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-17T17:19:45.249 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-17T17:19:48.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:19:48.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:51.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-17T17:19:55.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-17T17:19:55.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:19:56.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-09-17T17:19:57.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T17:19:57.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:20:03.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:20:03.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:20:06.209 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-17T17:20:08.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-17T17:20:08.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:20:09.478 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-17T17:20:09.917 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-17T17:20:09.917 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T17:20:09.917 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-09-17T17:20:09.933 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-508.el9.x86_64 2024-09-17T17:20:09.933 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T17:20:09.933 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T17:20:09.933 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T17:20:10.934 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T17:20:10.934 DEBUG:teuthology.orchestra.run.smithi026:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T17:20:11.001 INFO:teuthology.orchestra.run.smithi026.stdout:ttyS1 2024-09-17T17:20:11.021 DEBUG:teuthology.parallel:result is None 2024-09-17T17:20:21.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:20:21.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:20:24.285 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-17T17:20:27.257 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-17T17:20:27.915 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-17T17:20:27.915 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-508.el9.x86_64"... 2024-09-17T17:20:27.915 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-17T17:20:27.930 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-508.el9.x86_64 2024-09-17T17:20:27.930 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T17:20:27.930 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T17:20:27.930 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T17:20:28.931 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-17T17:20:28.932 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T17:20:28.992 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-09-17T17:20:29.014 DEBUG:teuthology.parallel:result is None 2024-09-17T17:20:29.068 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-17T17:20:29.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T17:20:29.481 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T17:20:29.482 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-17T17:20:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-508.el9.x86_64 2024-09-17T17:20:29.497 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T17:20:29.497 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T17:20:29.497 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T17:20:30.501 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T17:20:30.502 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T17:20:30.590 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-09-17T17:20:30.617 DEBUG:teuthology.parallel:result is None 2024-09-17T17:20:42.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:20:42.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:21:00.737 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-17T17:21:21.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:21:21.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:21:22.010 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-17T17:21:22.379 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:21:22.379 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-17T17:21:22.379 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-17T17:21:22.396 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-508.el9.x86_64 2024-09-17T17:21:22.396 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T17:21:22.396 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T17:21:22.396 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T17:21:23.397 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-17T17:21:23.397 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T17:21:23.473 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2024-09-17T17:21:23.496 DEBUG:teuthology.parallel:result is None 2024-09-17T17:21:23.496 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T17:21:23.507 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T17:21:23.507 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T17:21:23.510 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T17:21:23.513 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T17:21:23.516 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T17:21:23.535 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T17:21:23.542 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T17:21:23.577 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T17:21:23.579 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T17:21:23.582 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T17:21:23.585 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T17:21:23.588 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T17:21:23.622 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T17:21:23.628 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T17:21:23.629 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T17:21:23.650 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T17:21:23.653 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T17:21:23.655 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T17:21:23.684 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.686 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.686 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.691 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.696 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.697 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.698 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.702 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T17:21:23.704 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T17:21:23.712 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T17:21:23.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T17:21:23.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T17:21:23.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T17:21:23.745 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T17:21:23.772 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T17:21:23.786 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T17:21:23.786 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T17:21:23.810 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T17:21:23.813 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T17:21:23.815 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T17:21:23.831 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:23.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:23.965 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.095 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:21:24.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T17:21:24.167 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:24.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:24.266 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.397 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:21:24.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T17:21:24.466 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:24.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:24.567 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.697 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:21:24.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T17:21:24.765 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:24.810 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T17:21:24.862 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T17:21:24.991 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:21:24.991 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T17:21:25.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-17T17:21:25.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-09-17T17:21:25.064 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-17T17:21:25.067 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-17T17:21:25.113 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:21:25.115 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:21:25.117 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:21:25.146 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:21:25.623 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T17:21:25.631 INFO:teuthology.task.internal:Starting timer... 2024-09-17T17:21:25.632 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T17:21:25.676 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T17:21:25.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-17T17:21:25.780 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-17T17:21:25.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-09-17T17:21:25.831 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-09-17T17:21:25.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-09-17T17:21:25.882 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-09-17T17:21:25.884 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-17T17:21:25.955 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-17T17:21:25.957 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T17:21:25.957 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-17T17:21:25.981 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-17T17:21:25.981 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-09-17T17:21:26.007 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-09-17T17:21:26.007 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-09-17T17:21:26.034 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-09-17T17:21:26.034 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-17T17:21:26.063 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-17T17:21:26.063 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-09-17T17:21:26.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:21:26.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:21:26.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:21:26.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:21:26.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:21:26.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:21:26.154 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:21:26.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:21:26.182 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T17:21:26.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-17T17:21:26.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-09-17T17:21:26.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-09-17T17:21:26.282 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-17T17:21:26.312 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T17:21:26.322 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T17:21:26.325 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T17:21:26.446 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T17:21:26.461 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T17:21:26.462 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi026.front.sepia.ceph.com,smithi027.front.sepia.ceph.com,smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T17:25:27.968 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-09-17T17:25:27.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T17:25:27.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:25:28.048 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-17T17:25:28.085 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T17:25:28.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-17T17:25:28.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:25:28.163 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-17T17:25:28.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-17T17:25:28.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:25:28.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:25:28.324 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-17T17:25:28.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-17T17:25:28.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-17T17:25:28.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T17:25:28.494 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-17T17:25:28.569 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-17T17:25:28.570 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T17:25:28.580 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T17:25:28.581 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T17:25:28.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:25:28.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T17:25:28.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:25:28.586 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T17:25:28.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:25:28.589 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T17:25:28.589 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:25:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T17:25:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T17:25:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T17:25:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T17:25:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T17:25:28.638 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T17:25:28.639 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T17:25:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T17:25:28.661 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-17T17:25:28.663 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-09-17T17:25:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-09-17T17:25:28.673 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-17T17:25:28.675 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-09-17T17:25:28.678 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-09-17T17:25:28.678 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-17T17:25:28.690 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T17:25:28.690 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-17T17:25:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T17:25:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T17:25:28.703 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T17:25:28.704 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T17:25:28.706 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T17:25:28.709 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T17:25:28.717 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T17:25:28.762 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:25:28.765 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:25:28.765 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-17T17:25:28.766 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.766 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.766 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.766 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:25:28.767 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:25:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:25:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-09-17T17:25:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.769 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.770 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.771 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:25:28.771 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-17T17:25:28.771 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.771 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.771 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.771 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.772 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:25:28.774 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:25:28.775 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-17T17:25:28.775 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.775 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.775 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.775 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:25:28.776 INFO:teuthology.run_tasks:Running task install... 2024-09-17T17:25:28.784 DEBUG:teuthology.task.install:project ceph 2024-09-17T17:25:28.784 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}} 2024-09-17T17:25:28.784 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'} 2024-09-17T17:25:28.784 INFO:teuthology.task.install:Using flavor: default 2024-09-17T17:25:28.789 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T17:25:28.789 INFO:teuthology.task.install:extra packages: [] 2024-09-17T17:25:28.790 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'tag': None, 'wait_for_package': False} 2024-09-17T17:25:28.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:25:28.792 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'tag': None, 'wait_for_package': False} 2024-09-17T17:25:28.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:25:28.793 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'tag': None, 'wait_for_package': False} 2024-09-17T17:25:28.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:25:28.794 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'tag': None, 'wait_for_package': False} 2024-09-17T17:25:28.794 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:25:28.939 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/ 2024-09-17T17:25:28.940 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g1889a165 2024-09-17T17:25:28.948 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/ 2024-09-17T17:25:28.948 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g1889a165 2024-09-17T17:25:28.954 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/ 2024-09-17T17:25:28.954 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g1889a165 2024-09-17T17:25:28.957 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/ 2024-09-17T17:25:28.957 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g1889a165 2024-09-17T17:25:29.133 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T17:25:29.134 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:25:29.134 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T17:25:29.141 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T17:25:29.141 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:25:29.141 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T17:25:29.148 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T17:25:29.149 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:25:29.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T17:25:29.153 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T17:25:29.154 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:25:29.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T17:25:29.161 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T17:25:29.162 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1889a165b5128787ec693450a3d1c5df5d6dc203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T17:25:29.167 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T17:25:29.167 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1889a165b5128787ec693450a3d1c5df5d6dc203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T17:25:29.177 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T17:25:29.177 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1889a165b5128787ec693450a3d1c5df5d6dc203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T17:25:29.181 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T17:25:29.181 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1889a165b5128787ec693450a3d1c5df5d6dc203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T17:25:29.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T17:25:29.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T17:25:29.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T17:25:29.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T17:25:29.308 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T17:25:29.309 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T17:25:29.327 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T17:25:29.328 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T17:25:29.372 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-17T17:25:29.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-17T17:25:29.376 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-17T17:25:29.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-17T17:25:29.393 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-09-17T17:25:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-09-17T17:25:29.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-09-17T17:25:29.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-09-17T17:25:29.605 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-17T17:25:29.623 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-17T17:25:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:45 files removed 2024-09-17T17:25:29.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw 2024-09-17T17:25:29.639 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-09-17T17:25:29.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw 2024-09-17T17:25:29.655 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw 2024-09-17T17:25:29.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw 2024-09-17T17:25:30.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 284 kB/s | 84 kB 00:00 2024-09-17T17:25:30.247 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-09-17T17:25:30.251 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 283 kB/s | 84 kB 00:00 2024-09-17T17:25:30.263 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 268 kB/s | 84 kB 00:00 2024-09-17T17:25:30.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-17T17:25:30.523 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 105 kB/s | 24 kB 00:00 2024-09-17T17:25:30.526 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 105 kB/s | 24 kB 00:00 2024-09-17T17:25:30.555 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-17T17:25:30.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T17:25:30.792 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-17T17:25:30.793 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-17T17:25:30.831 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T17:25:31.215 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-17T17:25:31.431 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-17T17:25:31.694 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.3 MB 00:00 2024-09-17T17:25:35.782 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.3 MB 00:04 2024-09-17T17:25:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 20 MB 00:04 2024-09-17T17:25:37.353 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-17T17:25:37.908 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-17T17:25:41.208 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-17T17:25:42.837 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-17T17:25:42.901 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-17T17:25:44.030 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:01 2024-09-17T17:25:44.354 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-17T17:25:44.659 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.5 MB 00:02 2024-09-17T17:25:45.483 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:01 2024-09-17T17:25:46.951 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-17T17:25:47.905 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-17T17:25:49.744 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-17T17:25:49.848 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-09-17T17:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-17T17:25:52.083 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 11 M 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.5 M 2024-09-17T17:25:52.086 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.3 M 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 22 M 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.3.0-4975.g1889a165.el9 ceph 25 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 756 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T17:25:52.087 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 508 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 45 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 143 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.3.0-4975.g1889a165.el9 ceph 161 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.3.0-4975.g1889a165.el9 ceph 321 k 2024-09-17T17:25:52.088 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 305 k 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 99 k 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:25:52.089 INFO:teuthology.orchestra.run.smithi007.stdout:Install 31 Packages 2024-09-17T17:25:52.090 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-09-17T17:25:52.090 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:25:52.090 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 62 M 2024-09-17T17:25:52.090 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 11 M 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.5 M 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.3 M 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 22 M 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.3.0-4975.g1889a165.el9 ceph 25 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 756 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 508 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 45 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 143 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.3.0-4975.g1889a165.el9 ceph 161 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.3.0-4975.g1889a165.el9 ceph 321 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 305 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 99 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T17:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout:Install 31 Packages 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 2 Packages 2024-09-17T17:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 62 M 2024-09-17T17:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:25:53.511 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-17T17:25:54.504 INFO:teuthology.orchestra.run.smithi007.stdout:(1/33): ceph-base-19.3.0-4975.g1889a165.el9.x86 2.8 MB/s | 5.5 MB 00:01 2024-09-17T17:25:54.579 INFO:teuthology.orchestra.run.smithi007.stdout:(2/33): ceph-selinux-19.3.0-4975.g1889a165.el9. 336 kB/s | 25 kB 00:00 2024-09-17T17:25:54.787 INFO:teuthology.orchestra.run.smithi007.stdout:(3/33): libcephfs2-19.3.0-4975.g1889a165.el9.x8 3.5 MB/s | 756 kB 00:00 2024-09-17T17:25:54.938 INFO:teuthology.orchestra.run.smithi007.stdout:(4/33): libradosstriper1-19.3.0-4975.g1889a165. 3.3 MB/s | 508 kB 00:00 2024-09-17T17:25:55.321 INFO:teuthology.orchestra.run.smithi007.stdout:(5/33): ceph-radosgw-19.3.0-4975.g1889a165.el9. 3.9 MB/s | 11 MB 00:02 2024-09-17T17:25:55.332 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-17T17:25:55.596 INFO:teuthology.orchestra.run.smithi007.stdout:(6/33): ceph-common-19.3.0-4975.g1889a165.el9.x 7.1 MB/s | 22 MB 00:03 2024-09-17T17:25:55.622 INFO:teuthology.orchestra.run.smithi007.stdout:(7/33): python3-ceph-argparse-19.3.0-4975.g1889 150 kB/s | 45 kB 00:00 2024-09-17T17:25:55.672 INFO:teuthology.orchestra.run.smithi007.stdout:(8/33): python3-ceph-common-19.3.0-4975.g1889a1 1.9 MB/s | 143 kB 00:00 2024-09-17T17:25:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:(9/33): python3-cephfs-19.3.0-4975.g1889a165.el 2.1 MB/s | 161 kB 00:00 2024-09-17T17:25:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:(10/33): python3-rados-19.3.0-4975.g1889a165.el 3.8 MB/s | 321 kB 00:00 2024-09-17T17:25:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:(11/33): python3-rbd-19.3.0-4975.g1889a165.el9. 3.6 MB/s | 305 kB 00:00 2024-09-17T17:25:55.800 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:25:55.809 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 11 M 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.5 M 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.3 M 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 22 M 2024-09-17T17:25:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.3.0-4975.g1889a165.el9 ceph 25 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 756 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 508 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T17:25:55.811 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 45 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 143 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.3.0-4975.g1889a165.el9 ceph 161 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.3.0-4975.g1889a165.el9 ceph 321 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 305 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 99 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T17:25:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout:Install 31 Packages 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 2 Packages 2024-09-17T17:25:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:25:55.814 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 62 M 2024-09-17T17:25:55.814 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:25:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:(12/33): python3-rgw-19.3.0-4975.g1889a165.el9. 1.3 MB/s | 99 kB 00:00 2024-09-17T17:25:55.974 INFO:teuthology.orchestra.run.smithi007.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 557 kB/s | 107 kB 00:00 2024-09-17T17:25:55.999 INFO:teuthology.orchestra.run.smithi007.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.0 MB/s | 171 kB 00:00 2024-09-17T17:25:56.049 INFO:teuthology.orchestra.run.smithi007.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-09-17T17:25:56.083 INFO:teuthology.orchestra.run.smithi007.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-17T17:25:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:(17/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T17:25:56.275 INFO:teuthology.orchestra.run.smithi007.stdout:(18/33): librgw2-19.3.0-4975.g1889a165.el9.x86_ 4.1 MB/s | 5.5 MB 00:01 2024-09-17T17:25:56.300 INFO:teuthology.orchestra.run.smithi007.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-17T17:25:56.326 INFO:teuthology.orchestra.run.smithi007.stdout:(20/33): python3-prettytable-0.7.2-27.el9.noarc 315 kB/s | 42 kB 00:00 2024-09-17T17:25:56.351 INFO:teuthology.orchestra.run.smithi007.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 713 kB/s | 36 kB 00:00 2024-09-17T17:25:56.385 INFO:teuthology.orchestra.run.smithi007.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-09-17T17:25:56.435 INFO:teuthology.orchestra.run.smithi007.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.6 MB/s | 308 kB 00:00 2024-09-17T17:25:56.452 INFO:teuthology.orchestra.run.smithi007.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.7 MB/s | 26 kB 00:00 2024-09-17T17:25:56.469 INFO:teuthology.orchestra.run.smithi007.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-17T17:25:56.481 INFO:teuthology.orchestra.run.smithi026.stdout:(1/33): ceph-radosgw-19.3.0-4975.g1889a165.el9. 3.9 MB/s | 11 MB 00:02 2024-09-17T17:25:56.494 INFO:teuthology.orchestra.run.smithi007.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-17T17:25:56.528 INFO:teuthology.orchestra.run.smithi007.stdout:(27/33): qatlib-24.02.0-1.el9.x86_64.rpm 874 kB/s | 221 kB 00:00 2024-09-17T17:25:56.548 INFO:teuthology.orchestra.run.smithi026.stdout:(2/33): ceph-selinux-19.3.0-4975.g1889a165.el9. 379 kB/s | 25 kB 00:00 2024-09-17T17:25:56.569 INFO:teuthology.orchestra.run.smithi007.stdout:(28/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-17T17:25:56.645 INFO:teuthology.orchestra.run.smithi007.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-17T17:25:56.695 INFO:teuthology.orchestra.run.smithi007.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-17T17:25:56.707 INFO:teuthology.orchestra.run.smithi026.stdout:(3/33): ceph-base-19.3.0-4975.g1889a165.el9.x86 1.8 MB/s | 5.5 MB 00:02 2024-09-17T17:25:56.732 INFO:teuthology.orchestra.run.smithi026.stdout:(4/33): libcephfs2-19.3.0-4975.g1889a165.el9.x8 4.0 MB/s | 756 kB 00:00 2024-09-17T17:25:56.771 INFO:teuthology.orchestra.run.smithi007.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-17T17:25:56.982 INFO:teuthology.orchestra.run.smithi026.stdout:(5/33): libradosstriper1-19.3.0-4975.g1889a165. 1.8 MB/s | 508 kB 00:00 2024-09-17T17:25:57.091 INFO:teuthology.orchestra.run.smithi026.stdout:(6/33): python3-ceph-argparse-19.3.0-4975.g1889 417 kB/s | 45 kB 00:00 2024-09-17T17:25:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:(7/33): python3-ceph-common-19.3.0-4975.g1889a1 859 kB/s | 143 kB 00:00 2024-09-17T17:25:57.313 INFO:teuthology.orchestra.run.smithi007.stdout:(32/33): librbd1-19.3.0-4975.g1889a165.el9.x86_ 5.3 MB/s | 3.3 MB 00:00 2024-09-17T17:25:57.342 INFO:teuthology.orchestra.run.smithi026.stdout:(8/33): librgw2-19.3.0-4975.g1889a165.el9.x86_6 9.0 MB/s | 5.5 MB 00:00 2024-09-17T17:25:57.409 INFO:teuthology.orchestra.run.smithi026.stdout:(9/33): python3-cephfs-19.3.0-4975.g1889a165.el 1.0 MB/s | 161 kB 00:00 2024-09-17T17:25:57.434 INFO:teuthology.orchestra.run.smithi026.stdout:(10/33): python3-rados-19.3.0-4975.g1889a165.el 3.4 MB/s | 321 kB 00:00 2024-09-17T17:25:57.501 INFO:teuthology.orchestra.run.smithi026.stdout:(11/33): python3-rgw-19.3.0-4975.g1889a165.el9. 1.5 MB/s | 99 kB 00:00 2024-09-17T17:25:57.554 INFO:teuthology.orchestra.run.smithi007.stdout:(33/33): librados2-19.3.0-4975.g1889a165.el9.x8 3.8 MB/s | 3.5 MB 00:00 2024-09-17T17:25:57.559 INFO:teuthology.orchestra.run.smithi026.stdout:(12/33): python3-rbd-19.3.0-4975.g1889a165.el9. 2.0 MB/s | 305 kB 00:00 2024-09-17T17:25:57.560 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:25:57.560 INFO:teuthology.orchestra.run.smithi007.stdout:Total 11 MB/s | 62 MB 00:05 2024-09-17T17:25:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:(1/33): ceph-base-19.3.0-4975.g1889a165.el9.x86 3.8 MB/s | 5.5 MB 00:01 2024-09-17T17:25:57.675 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:25:57.717 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:25:57.717 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:25:57.721 INFO:teuthology.orchestra.run.smithi027.stdout:(2/33): ceph-selinux-19.3.0-4975.g1889a165.el9. 433 kB/s | 25 kB 00:00 2024-09-17T17:25:57.830 INFO:teuthology.orchestra.run.smithi027.stdout:(3/33): libcephfs2-19.3.0-4975.g1889a165.el9.x8 6.8 MB/s | 756 kB 00:00 2024-09-17T17:25:57.938 INFO:teuthology.orchestra.run.smithi027.stdout:(4/33): libradosstriper1-19.3.0-4975.g1889a165. 4.6 MB/s | 508 kB 00:00 2024-09-17T17:25:57.967 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 19 kB 00:01 2024-09-17T17:25:58.226 INFO:teuthology.orchestra.run.smithi026.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 147 kB/s | 107 kB 00:00 2024-09-17T17:25:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 222 kB/s | 171 kB 00:00 2024-09-17T17:25:58.339 INFO:teuthology.orchestra.run.smithi027.stdout:(5/33): librgw2-19.3.0-4975.g1889a165.el9.x86_6 14 MB/s | 5.5 MB 00:00 2024-09-17T17:25:58.380 INFO:teuthology.orchestra.run.smithi027.stdout:(6/33): python3-ceph-argparse-19.3.0-4975.g1889 1.1 MB/s | 45 kB 00:00 2024-09-17T17:25:58.425 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:25:58.425 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:25:58.431 INFO:teuthology.orchestra.run.smithi027.stdout:(7/33): python3-ceph-common-19.3.0-4975.g1889a1 2.8 MB/s | 143 kB 00:00 2024-09-17T17:25:58.481 INFO:teuthology.orchestra.run.smithi027.stdout:(8/33): python3-cephfs-19.3.0-4975.g1889a165.el 3.2 MB/s | 161 kB 00:00 2024-09-17T17:25:58.631 INFO:teuthology.orchestra.run.smithi027.stdout:(9/33): ceph-radosgw-19.3.0-4975.g1889a165.el9. 4.5 MB/s | 11 MB 00:02 2024-09-17T17:25:58.665 INFO:teuthology.orchestra.run.smithi027.stdout:(10/33): python3-rados-19.3.0-4975.g1889a165.el 1.7 MB/s | 321 kB 00:00 2024-09-17T17:25:58.715 INFO:teuthology.orchestra.run.smithi027.stdout:(11/33): python3-rbd-19.3.0-4975.g1889a165.el9. 3.6 MB/s | 305 kB 00:00 2024-09-17T17:25:58.741 INFO:teuthology.orchestra.run.smithi027.stdout:(12/33): python3-rgw-19.3.0-4975.g1889a165.el9. 1.3 MB/s | 99 kB 00:00 2024-09-17T17:25:58.762 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-17T17:25:58.902 INFO:teuthology.orchestra.run.smithi026.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-09-17T17:25:58.993 INFO:teuthology.orchestra.run.smithi026.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 209 kB/s | 160 kB 00:00 2024-09-17T17:25:59.399 INFO:teuthology.orchestra.run.smithi027.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 156 kB/s | 107 kB 00:00 2024-09-17T17:25:59.458 INFO:teuthology.orchestra.run.smithi027.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 238 kB/s | 171 kB 00:00 2024-09-17T17:25:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:(17/33): ceph-common-19.3.0-4975.g1889a165.el9. 3.6 MB/s | 22 MB 00:05 2024-09-17T17:25:59.944 INFO:teuthology.orchestra.run.smithi026.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 636 kB/s | 662 kB 00:01 2024-09-17T17:25:59.994 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:26:00.025 INFO:teuthology.orchestra.run.smithi027.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-09-17T17:26:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T17:26:00.108 INFO:teuthology.orchestra.run.smithi027.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 226 kB/s | 160 kB 00:00 2024-09-17T17:26:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T17:26:00.119 INFO:teuthology.orchestra.run.smithi026.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 260 kB/s | 292 kB 00:01 2024-09-17T17:26:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T17:26:00.157 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:00.181 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:00.311 INFO:teuthology.orchestra.run.smithi026.stdout:(20/33): python3-prettytable-0.7.2-27.el9.noarc 68 kB/s | 42 kB 00:00 2024-09-17T17:26:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:00.695 INFO:teuthology.orchestra.run.smithi026.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 294 kB/s | 221 kB 00:00 2024-09-17T17:26:00.728 INFO:teuthology.orchestra.run.smithi026.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 59 kB/s | 36 kB 00:00 2024-09-17T17:26:00.770 INFO:teuthology.orchestra.run.smithi026.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-17T17:26:00.795 INFO:teuthology.orchestra.run.smithi026.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-17T17:26:00.821 INFO:teuthology.orchestra.run.smithi026.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T17:26:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T17:26:00.846 INFO:teuthology.orchestra.run.smithi026.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-17T17:26:00.883 INFO:teuthology.orchestra.run.smithi027.stdout:(17/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-09-17T17:26:00.896 INFO:teuthology.orchestra.run.smithi026.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-17T17:26:00.928 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:00.930 INFO:teuthology.orchestra.run.smithi026.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T17:26:00.947 INFO:teuthology.orchestra.run.smithi026.stdout:(29/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-09-17T17:26:00.975 INFO:teuthology.orchestra.run.smithi027.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 697 kB/s | 662 kB 00:00 2024-09-17T17:26:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:01.014 INFO:teuthology.orchestra.run.smithi026.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-17T17:26:01.097 INFO:teuthology.orchestra.run.smithi026.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-17T17:26:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 8/35 2024-09-17T17:26:01.138 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:01.238 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T17:26:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 11/35 2024-09-17T17:26:01.292 INFO:teuthology.orchestra.run.smithi027.stdout:(19/33): ceph-common-19.3.0-4975.g1889a165.el9. 4.3 MB/s | 22 MB 00:05 2024-09-17T17:26:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:01.463 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:01.497 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T17:26:01.498 INFO:teuthology.orchestra.run.smithi026.stdout:(32/33): librbd1-19.3.0-4975.g1889a165.el9.x86_ 6.7 MB/s | 3.3 MB 00:00 2024-09-17T17:26:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T17:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout:(33/33): librados2-19.3.0-4975.g1889a165.el9.x8 4.9 MB/s | 3.5 MB 00:00 2024-09-17T17:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout:Total 7.5 MB/s | 62 MB 00:08 2024-09-17T17:26:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T17:26:01.721 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T17:26:01.743 INFO:teuthology.orchestra.run.smithi027.stdout:(20/33): python3-prettytable-0.7.2-27.el9.noarc 49 kB/s | 42 kB 00:00 2024-09-17T17:26:01.756 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T17:26:01.773 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:26:01.785 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T17:26:01.816 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:26:01.816 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:26:01.818 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T17:26:01.918 INFO:teuthology.orchestra.run.smithi027.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 58 kB/s | 36 kB 00:00 2024-09-17T17:26:01.968 INFO:teuthology.orchestra.run.smithi027.stdout:(22/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.1 MB/s | 308 kB 00:00 2024-09-17T17:26:01.993 INFO:teuthology.orchestra.run.smithi027.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 217 kB/s | 221 kB 00:01 2024-09-17T17:26:02.044 INFO:teuthology.orchestra.run.smithi027.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-17T17:26:02.073 INFO:teuthology.orchestra.run.smithi027.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T17:26:02.094 INFO:teuthology.orchestra.run.smithi027.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-17T17:26:02.161 INFO:teuthology.orchestra.run.smithi027.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-17T17:26:02.187 INFO:teuthology.orchestra.run.smithi027.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 7.4 MB/s | 191 kB 00:00 2024-09-17T17:26:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T17:26:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:02.253 INFO:teuthology.orchestra.run.smithi027.stdout:(29/33): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-17T17:26:02.304 INFO:teuthology.orchestra.run.smithi027.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-17T17:26:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:02.378 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 24/35 2024-09-17T17:26:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T17:26:02.429 INFO:teuthology.orchestra.run.smithi027.stdout:(31/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-09-17T17:26:02.534 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:26:02.535 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:26:02.554 INFO:teuthology.orchestra.run.smithi027.stdout:(32/33): librados2-19.3.0-4975.g1889a165.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-17T17:26:02.838 INFO:teuthology.orchestra.run.smithi027.stdout:(33/33): librbd1-19.3.0-4975.g1889a165.el9.x86_ 6.1 MB/s | 3.3 MB 00:00 2024-09-17T17:26:02.843 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:02.843 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.8 MB/s | 62 MB 00:07 2024-09-17T17:26:02.941 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T17:26:02.958 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:26:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:03.002 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:26:03.002 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:26:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:03.128 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 28/35 2024-09-17T17:26:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 29/35 2024-09-17T17:26:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:26:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:26:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:26:04.193 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-17T17:26:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T17:26:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T17:26:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T17:26:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T17:26:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 8/35 2024-09-17T17:26:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:05.256 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:26:05.304 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T17:26:05.341 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T17:26:05.368 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T17:26:05.385 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 11/35 2024-09-17T17:26:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T17:26:05.424 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:05.452 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:05.493 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:05.558 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:05.592 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T17:26:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:05.652 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:05.708 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:05.724 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T17:26:05.760 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T17:26:05.801 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T17:26:05.833 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T17:26:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T17:26:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:05.932 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T17:26:06.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T17:26:06.170 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:06.272 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T17:26:06.275 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 8/35 2024-09-17T17:26:06.400 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:06.403 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:06.403 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T17:26:06.403 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T17:26:06.403 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:06.430 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 24/35 2024-09-17T17:26:06.433 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:06.437 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 11 M 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-17T17:26:06.456 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.5 M 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.3 M 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 22 M 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.3.0-4975.g1889a165.el9 ceph 25 k 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 756 k 2024-09-17T17:26:06.457 INFO:teuthology.orchestra.run.smithi102.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 508 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 45 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 143 k 2024-09-17T17:26:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.3.0-4975.g1889a165.el9 ceph 161 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.3.0-4975.g1889a165.el9 ceph 321 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 305 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 99 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:06.459 INFO:teuthology.orchestra.run.smithi102.stdout:Install 31 Packages 2024-09-17T17:26:06.460 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 2 Packages 2024-09-17T17:26:06.460 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:06.460 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 62 M 2024-09-17T17:26:06.460 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:26:06.467 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T17:26:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T17:26:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 11/35 2024-09-17T17:26:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T17:26:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T17:26:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T17:26:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T17:26:06.985 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T17:26:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T17:26:07.011 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T17:26:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:07.083 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T17:26:07.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 28/35 2024-09-17T17:26:07.205 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 29/35 2024-09-17T17:26:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T17:26:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 24/35 2024-09-17T17:26:07.621 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T17:26:07.837 INFO:teuthology.orchestra.run.smithi102.stdout:(1/33): ceph-base-19.3.0-4975.g1889a165.el9.x86 6.0 MB/s | 5.5 MB 00:00 2024-09-17T17:26:07.878 INFO:teuthology.orchestra.run.smithi102.stdout:(2/33): ceph-selinux-19.3.0-4975.g1889a165.el9. 609 kB/s | 25 kB 00:00 2024-09-17T17:26:07.979 INFO:teuthology.orchestra.run.smithi102.stdout:(3/33): libcephfs2-19.3.0-4975.g1889a165.el9.x8 7.4 MB/s | 756 kB 00:00 2024-09-17T17:26:08.062 INFO:teuthology.orchestra.run.smithi102.stdout:(4/33): libradosstriper1-19.3.0-4975.g1889a165. 6.0 MB/s | 508 kB 00:00 2024-09-17T17:26:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T17:26:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:08.213 INFO:teuthology.orchestra.run.smithi102.stdout:(5/33): ceph-radosgw-19.3.0-4975.g1889a165.el9. 8.4 MB/s | 11 MB 00:01 2024-09-17T17:26:08.238 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:08.246 INFO:teuthology.orchestra.run.smithi102.stdout:(6/33): python3-ceph-argparse-19.3.0-4975.g1889 1.3 MB/s | 45 kB 00:00 2024-09-17T17:26:08.297 INFO:teuthology.orchestra.run.smithi102.stdout:(7/33): python3-ceph-common-19.3.0-4975.g1889a1 2.8 MB/s | 143 kB 00:00 2024-09-17T17:26:08.318 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 28/35 2024-09-17T17:26:08.343 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 29/35 2024-09-17T17:26:08.355 INFO:teuthology.orchestra.run.smithi102.stdout:(8/33): python3-cephfs-19.3.0-4975.g1889a165.el 2.7 MB/s | 161 kB 00:00 2024-09-17T17:26:08.422 INFO:teuthology.orchestra.run.smithi102.stdout:(9/33): python3-rados-19.3.0-4975.g1889a165.el9 4.7 MB/s | 321 kB 00:00 2024-09-17T17:26:08.489 INFO:teuthology.orchestra.run.smithi102.stdout:(10/33): python3-rbd-19.3.0-4975.g1889a165.el9. 4.5 MB/s | 305 kB 00:00 2024-09-17T17:26:08.564 INFO:teuthology.orchestra.run.smithi102.stdout:(11/33): librgw2-19.3.0-4975.g1889a165.el9.x86_ 11 MB/s | 5.5 MB 00:00 2024-09-17T17:26:08.589 INFO:teuthology.orchestra.run.smithi102.stdout:(12/33): python3-rgw-19.3.0-4975.g1889a165.el9. 992 kB/s | 99 kB 00:00 2024-09-17T17:26:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:(13/33): ceph-common-19.3.0-4975.g1889a165.el9. 12 MB/s | 22 MB 00:01 2024-09-17T17:26:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 388 kB/s | 107 kB 00:00 2024-09-17T17:26:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-17T17:26:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 435 kB/s | 171 kB 00:00 2024-09-17T17:26:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 738 kB/s | 160 kB 00:00 2024-09-17T17:26:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-17T17:26:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-17T17:26:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:(20/33): qatlib-service-24.02.0-1.el9.x86_64.rp 1.1 MB/s | 36 kB 00:00 2024-09-17T17:26:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-17T17:26:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-17T17:26:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-17T17:26:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-17T17:26:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:(25/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-17T17:26:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-17T17:26:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 19 MB/s | 839 kB 00:00 2024-09-17T17:26:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:(28/33): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-09-17T17:26:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-17T17:26:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-17T17:26:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-17T17:26:09.631 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:09.667 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:(32/33): librbd1-19.3.0-4975.g1889a165.el9.x86_ 16 MB/s | 3.3 MB 00:00 2024-09-17T17:26:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:(33/33): librados2-19.3.0-4975.g1889a165.el9.x8 9.6 MB/s | 3.5 MB 00:00 2024-09-17T17:26:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:Total 19 MB/s | 62 MB 00:03 2024-09-17T17:26:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:26:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:26:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:26:10.259 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:10.304 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:10.383 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T17:26:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T17:26:10.384 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:26:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:26:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:11.420 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:11.460 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:11.551 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:11.551 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T17:26:11.551 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T17:26:11.551 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:11.586 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:12.086 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:26:12.168 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T17:26:12.209 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T17:26:12.234 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T17:26:12.256 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:12.281 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T17:26:12.475 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:12.528 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T17:26:12.933 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T17:26:12.994 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:13.087 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:13.178 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 8/35 2024-09-17T17:26:13.204 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:13.267 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T17:26:13.303 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T17:26:13.355 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 11/35 2024-09-17T17:26:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:13.448 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:13.512 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T17:26:13.546 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T17:26:13.653 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T17:26:13.691 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T17:26:13.730 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T17:26:13.762 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T17:26:13.800 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T17:26:13.841 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:13.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T17:26:14.194 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T17:26:14.229 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:14.320 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T17:26:14.351 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 24/35 2024-09-17T17:26:14.387 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T17:26:14.875 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T17:26:14.898 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:14.970 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T17:26:15.039 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 28/35 2024-09-17T17:26:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 29/35 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-17T17:26:17.133 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:17.390 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:17.430 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:17.995 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T17:26:18.034 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:18.127 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T17:26:18.127 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T17:26:18.128 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T17:26:18.128 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:18.169 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:18.203 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:18.203 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T17:26:18.203 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:18.204 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:18.304 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:18.304 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 1/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 3/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 4/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 5/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 6/35 2024-09-17T17:26:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 8/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 9/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 10/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 11/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 13/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T17:26:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T17:26:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T17:26:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 34/35 2024-09-17T17:26:20.503 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:20.503 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T17:26:20.504 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.505 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:20.506 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:20.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-test 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-09-17T17:26:21.099 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:21.482 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:21.631 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:21.631 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.3.0-4975.g1889a165.el9 ceph 46 M 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.3.0-4975.g1889a165.el9 ceph 162 k 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:26:21.632 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:21.633 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-17T17:26:21.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:21.633 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 46 M 2024-09-17T17:26:21.633 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 201 M 2024-09-17T17:26:21.633 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:26:21.763 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:21.763 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-09-17T17:26:21.763 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-09-17T17:26:21.763 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-09-17T17:26:21.764 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-09-17T17:26:21.764 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-09-17T17:26:21.764 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-09-17T17:26:21.764 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-09-17T17:26:21.764 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:22.139 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:22.142 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-17T17:26:22.166 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:22.167 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:22.167 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T17:26:22.167 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:22.167 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:22.167 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:22.168 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): libcephsqlite-19.3.0-4975.g1889a165.el9. 653 kB/s | 162 kB 00:00 2024-09-17T17:26:22.169 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:22.193 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-17T17:26:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:22.804 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:22.831 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:22.831 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:22.831 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T17:26:22.831 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:22.831 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:22.831 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:22.833 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:22.924 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:22.924 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:23.410 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): ceph-test-19.3.0-4975.g1889a165.el9.x86_ 31 MB/s | 46 MB 00:01 2024-09-17T17:26:23.412 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:23.412 INFO:teuthology.orchestra.run.smithi007.stdout:Total 26 MB/s | 46 MB 00:01 2024-09-17T17:26:23.441 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:26:23.464 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:26:23.464 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:26:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 1/35 2024-09-17T17:26:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2/35 2024-09-17T17:26:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 3/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 4/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 5/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 6/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 8/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 9/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 10/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 11/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 13/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T17:26:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T17:26:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T17:26:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T17:26:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T17:26:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T17:26:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 34/35 2024-09-17T17:26:23.871 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:26:23.871 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 1/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 3/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 4/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 5/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 6/35 2024-09-17T17:26:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 8/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 9/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 10/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 11/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 13/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T17:26:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T17:26:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T17:26:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 34/35 2024-09-17T17:26:24.420 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:26:24.503 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T17:26:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T17:26:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:24.556 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-09-17T17:26:24.556 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.558 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:24.559 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:24.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-test 2024-09-17T17:26:25.188 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:25.188 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:25.188 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-17T17:26:25.188 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.188 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T17:26:25.189 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T17:26:25.190 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:25.191 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:25.508 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:34 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:25.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-test 2024-09-17T17:26:25.653 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.3.0-4975.g1889a165.el9 ceph 46 M 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:26:25.654 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.3.0-4975.g1889a165.el9 ceph 162 k 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout:Install 4 Packages 2024-09-17T17:26:25.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:25.656 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 46 M 2024-09-17T17:26:25.656 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 201 M 2024-09-17T17:26:25.656 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:26:25.879 INFO:teuthology.orchestra.run.smithi026.stdout:(1/4): libcephsqlite-19.3.0-4975.g1889a165.el9. 813 kB/s | 162 kB 00:00 2024-09-17T17:26:26.112 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:26.258 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.3.0-4975.g1889a165.el9 ceph 46 M 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.3.0-4975.g1889a165.el9 ceph 162 k 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:26:26.259 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:26.260 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-09-17T17:26:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:26.260 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 46 M 2024-09-17T17:26:26.260 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 201 M 2024-09-17T17:26:26.260 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:26:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-17T17:26:26.496 INFO:teuthology.orchestra.run.smithi026.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-17T17:26:26.529 INFO:teuthology.orchestra.run.smithi026.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 358 kB/s | 304 kB 00:00 2024-09-17T17:26:26.530 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 960 kB/s | 64 kB 00:00 2024-09-17T17:26:26.721 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): libcephsqlite-19.3.0-4975.g1889a165.el9. 388 kB/s | 162 kB 00:00 2024-09-17T17:26:27.105 INFO:teuthology.orchestra.run.smithi026.stdout:(4/4): ceph-test-19.3.0-4975.g1889a165.el9.x86_ 32 MB/s | 46 MB 00:01 2024-09-17T17:26:27.107 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:27.107 INFO:teuthology.orchestra.run.smithi026.stdout:Total 32 MB/s | 46 MB 00:01 2024-09-17T17:26:27.136 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:26:27.159 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:26:27.159 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:26:27.554 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:26:27.554 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:26:27.913 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): ceph-test-19.3.0-4975.g1889a165.el9.x86_ 28 MB/s | 46 MB 00:01 2024-09-17T17:26:27.915 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:27.916 INFO:teuthology.orchestra.run.smithi027.stdout:Total 28 MB/s | 46 MB 00:01 2024-09-17T17:26:27.945 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:26:27.967 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:26:27.967 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:26:28.083 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:26:28.173 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T17:26:28.227 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T17:26:28.256 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:28.343 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:28.343 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-17T17:26:28.343 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-17T17:26:28.344 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-17T17:26:28.344 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-17T17:26:28.344 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-17T17:26:28.344 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-17T17:26:28.344 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-17T17:26:28.344 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:28.355 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:26:28.355 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:26:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:26:28.966 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T17:26:29.011 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T17:26:29.040 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:29.175 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:29.363 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:29.388 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T17:26:29.388 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:29.389 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T17:26:29.389 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:29.389 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T17:26:29.389 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:29.390 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:29.474 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T17:26:29.962 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:30.631 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:30.631 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 1/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 3/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 4/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 5/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 6/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 8/35 2024-09-17T17:26:30.632 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 9/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 10/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 11/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 13/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T17:26:30.633 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T17:26:30.634 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T17:26:30.635 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T17:26:30.636 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T17:26:30.636 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T17:26:30.636 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T17:26:30.636 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T17:26:30.636 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T17:26:30.636 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 34/35 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.521 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.522 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T17:26:31.523 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T17:26:31.524 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T17:26:31.524 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:31.524 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:26:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 1/4 2024-09-17T17:26:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2/4 2024-09-17T17:26:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T17:26:31.850 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-test 2024-09-17T17:26:32.131 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:32.132 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:32.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph 2024-09-17T17:26:32.431 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:26:32.564 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:26:32.564 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:32.564 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:26:32.564 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:32.564 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.3.0-4975.g1889a165.el9 ceph 46 M 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.3.0-4975.g1889a165.el9 ceph 162 k 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-17T17:26:32.565 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:32.566 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 46 M 2024-09-17T17:26:32.566 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 201 M 2024-09-17T17:26:32.566 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:26:32.795 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): libcephsqlite-19.3.0-4975.g1889a165.el9. 781 kB/s | 162 kB 00:00 2024-09-17T17:26:32.820 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-17T17:26:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:32.929 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 477 kB/s | 64 kB 00:00 2024-09-17T17:26:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:33.028 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:44 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:33.174 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.3.0-4975.g1889a165.el9 ceph 6.6 k 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.3.0-4975.g1889a165.el9 ceph 2.4 M 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.3.0-4975.g1889a165.el9 ceph 1.5 M 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 291 k 2024-09-17T17:26:33.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.3.0-4975.g1889a165.el9 ceph 4.8 M 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 17 M 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T17:26:33.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T17:26:33.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T17:26:33.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout:Install 37 Packages 2024-09-17T17:26:33.181 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:33.182 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 31 M 2024-09-17T17:26:33.182 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 109 M 2024-09-17T17:26:33.182 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:26:33.650 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:33.733 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:33.890 INFO:teuthology.orchestra.run.smithi007.stdout:(1/37): ceph-19.3.0-4975.g1889a165.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-17T17:26:34.012 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): ceph-test-19.3.0-4975.g1889a165.el9.x86_ 32 MB/s | 46 MB 00:01 2024-09-17T17:26:34.015 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:34.015 INFO:teuthology.orchestra.run.smithi102.stdout:Total 32 MB/s | 46 MB 00:01 2024-09-17T17:26:34.042 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:26:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:(2/37): ceph-mgr-19.3.0-4975.g1889a165.el9.x86_ 5.3 MB/s | 1.5 MB 00:00 2024-09-17T17:26:34.062 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:26:34.062 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:26:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:(3/37): ceph-mds-19.3.0-4975.g1889a165.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-17T17:26:34.233 INFO:teuthology.orchestra.run.smithi007.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g1889 3.1 MB/s | 291 kB 00:00 2024-09-17T17:26:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:(5/37): ceph-mon-19.3.0-4975.g1889a165.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-17T17:26:34.425 INFO:teuthology.orchestra.run.smithi007.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-17T17:26:34.429 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:26:34.430 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:26:34.467 INFO:teuthology.orchestra.run.smithi007.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-17T17:26:34.642 INFO:teuthology.orchestra.run.smithi007.stdout:(8/37): ceph-osd-19.3.0-4975.g1889a165.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-17T17:26:34.667 INFO:teuthology.orchestra.run.smithi007.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 674 kB/s | 135 kB 00:00 2024-09-17T17:26:34.718 INFO:teuthology.orchestra.run.smithi007.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 3.2 MB/s | 1.2 MB 00:00 2024-09-17T17:26:34.743 INFO:teuthology.orchestra.run.smithi007.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-17T17:26:34.776 INFO:teuthology.orchestra.run.smithi007.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 931 kB/s | 125 kB 00:00 2024-09-17T17:26:34.885 INFO:teuthology.orchestra.run.smithi007.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-09-17T17:26:34.910 INFO:teuthology.orchestra.run.smithi007.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 979 kB/s | 188 kB 00:00 2024-09-17T17:26:34.927 INFO:teuthology.orchestra.run.smithi007.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 935 kB/s | 172 kB 00:00 2024-09-17T17:26:34.935 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:26:34.961 INFO:teuthology.orchestra.run.smithi007.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T17:26:34.978 INFO:teuthology.orchestra.run.smithi007.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 835 kB/s | 77 kB 00:00 2024-09-17T17:26:35.013 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T17:26:35.028 INFO:teuthology.orchestra.run.smithi007.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-09-17T17:26:35.053 INFO:teuthology.orchestra.run.smithi007.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-17T17:26:35.066 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T17:26:35.089 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:35.104 INFO:teuthology.orchestra.run.smithi007.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-17T17:26:35.137 INFO:teuthology.orchestra.run.smithi007.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 400 kB/s | 43 kB 00:00 2024-09-17T17:26:35.163 INFO:teuthology.orchestra.run.smithi007.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 468 kB/s | 11 kB 00:00 2024-09-17T17:26:35.188 INFO:teuthology.orchestra.run.smithi007.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 710 kB/s | 18 kB 00:00 2024-09-17T17:26:35.255 INFO:teuthology.orchestra.run.smithi007.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 345 kB/s | 23 kB 00:00 2024-09-17T17:26:35.294 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:35.295 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 1/4 2024-09-17T17:26:35.295 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2/4 2024-09-17T17:26:35.295 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T17:26:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 586 kB/s | 172 kB 00:00 2024-09-17T17:26:35.372 INFO:teuthology.orchestra.run.smithi007.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 167 kB/s | 19 kB 00:00 2024-09-17T17:26:35.397 INFO:teuthology.orchestra.run.smithi007.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-17T17:26:35.423 INFO:teuthology.orchestra.run.smithi007.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-17T17:26:35.449 INFO:teuthology.orchestra.run.smithi007.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T17:26:35.473 INFO:teuthology.orchestra.run.smithi007.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-09-17T17:26:35.498 INFO:teuthology.orchestra.run.smithi007.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-17T17:26:35.524 INFO:teuthology.orchestra.run.smithi007.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-17T17:26:35.549 INFO:teuthology.orchestra.run.smithi007.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-17T17:26:35.574 INFO:teuthology.orchestra.run.smithi007.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-17T17:26:35.616 INFO:teuthology.orchestra.run.smithi007.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-17T17:26:35.642 INFO:teuthology.orchestra.run.smithi007.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 297 kB/s | 20 kB 00:00 2024-09-17T17:26:35.667 INFO:teuthology.orchestra.run.smithi007.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-17T17:26:35.667 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T17:26:35.667 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:35.668 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:35.671 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:35.672 INFO:teuthology.orchestra.run.smithi007.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-17T17:26:35.830 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:26:35.875 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:26:35.875 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:26:35.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph 2024-09-17T17:26:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 1/4 2024-09-17T17:26:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2/4 2024-09-17T17:26:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T17:26:36.371 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:26:36.371 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:26:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:36.422 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:36.572 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:45 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:36.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph 2024-09-17T17:26:36.722 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:36.724 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.3.0-4975.g1889a165.el9 ceph 6.6 k 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.3.0-4975.g1889a165.el9 ceph 2.4 M 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.3.0-4975.g1889a165.el9 ceph 1.5 M 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 291 k 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.3.0-4975.g1889a165.el9 ceph 4.8 M 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 17 M 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T17:26:36.725 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T17:26:36.726 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T17:26:36.727 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout:Install 37 Packages 2024-09-17T17:26:36.728 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:36.729 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 31 M 2024-09-17T17:26:36.729 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 109 M 2024-09-17T17:26:36.729 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:26:37.205 INFO:teuthology.orchestra.run.smithi026.stdout:(1/37): ceph-19.3.0-4975.g1889a165.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-17T17:26:37.314 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:44 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:37.372 INFO:teuthology.orchestra.run.smithi026.stdout:(2/37): ceph-mgr-19.3.0-4975.g1889a165.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-17T17:26:37.431 INFO:teuthology.orchestra.run.smithi026.stdout:(3/37): ceph-mds-19.3.0-4975.g1889a165.el9.x86_ 6.6 MB/s | 2.4 MB 00:00 2024-09-17T17:26:37.460 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.3.0-4975.g1889a165.el9 ceph 6.6 k 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:26:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.3.0-4975.g1889a165.el9 ceph 2.4 M 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.3.0-4975.g1889a165.el9 ceph 1.5 M 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 291 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.3.0-4975.g1889a165.el9 ceph 4.8 M 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 17 M 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T17:26:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T17:26:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:26:37.465 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:37.466 INFO:teuthology.orchestra.run.smithi027.stdout:Install 37 Packages 2024-09-17T17:26:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:37.466 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 31 M 2024-09-17T17:26:37.466 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 109 M 2024-09-17T17:26:37.466 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:26:37.514 INFO:teuthology.orchestra.run.smithi026.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g1889 3.4 MB/s | 291 kB 00:00 2024-09-17T17:26:37.606 INFO:teuthology.orchestra.run.smithi026.stdout:(5/37): ceph-mon-19.3.0-4975.g1889a165.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-17T17:26:37.757 INFO:teuthology.orchestra.run.smithi026.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-17T17:26:37.764 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:26:37.807 INFO:teuthology.orchestra.run.smithi026.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-17T17:26:37.820 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T17:26:37.863 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T17:26:37.897 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T17:26:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T17:26:38.007 INFO:teuthology.orchestra.run.smithi026.stdout:(8/37): ceph-osd-19.3.0-4975.g1889a165.el9.x86_ 27 MB/s | 17 MB 00:00 2024-09-17T17:26:38.022 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T17:26:38.032 INFO:teuthology.orchestra.run.smithi026.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 600 kB/s | 135 kB 00:00 2024-09-17T17:26:38.055 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T17:26:38.056 INFO:teuthology.orchestra.run.smithi027.stdout:(1/37): ceph-19.3.0-4975.g1889a165.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-17T17:26:38.066 INFO:teuthology.orchestra.run.smithi026.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-09-17T17:26:38.091 INFO:teuthology.orchestra.run.smithi026.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-17T17:26:38.150 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T17:26:38.150 INFO:teuthology.orchestra.run.smithi026.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 876 kB/s | 125 kB 00:00 2024-09-17T17:26:38.231 INFO:teuthology.orchestra.run.smithi027.stdout:(2/37): ceph-mgr-19.3.0-4975.g1889a165.el9.x86_ 5.0 MB/s | 1.5 MB 00:00 2024-09-17T17:26:38.233 INFO:teuthology.orchestra.run.smithi026.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-17T17:26:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T17:26:38.259 INFO:teuthology.orchestra.run.smithi026.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-17T17:26:38.284 INFO:teuthology.orchestra.run.smithi026.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 260 kB/s | 35 kB 00:00 2024-09-17T17:26:38.290 INFO:teuthology.orchestra.run.smithi027.stdout:(3/37): ceph-mds-19.3.0-4975.g1889a165.el9.x86_ 6.5 MB/s | 2.4 MB 00:00 2024-09-17T17:26:38.301 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T17:26:38.309 INFO:teuthology.orchestra.run.smithi026.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-17T17:26:38.334 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T17:26:38.334 INFO:teuthology.orchestra.run.smithi026.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-17T17:26:38.364 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T17:26:38.365 INFO:teuthology.orchestra.run.smithi027.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g1889 3.8 MB/s | 291 kB 00:00 2024-09-17T17:26:38.376 INFO:teuthology.orchestra.run.smithi026.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-17T17:26:38.395 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T17:26:38.402 INFO:teuthology.orchestra.run.smithi026.stdout:(19/37): python3-toml-0.10.2-6.el9.noarch.rpm 354 kB/s | 42 kB 00:00 2024-09-17T17:26:38.427 INFO:teuthology.orchestra.run.smithi026.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-17T17:26:38.429 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T17:26:38.458 INFO:teuthology.orchestra.run.smithi027.stdout:(5/37): ceph-mon-19.3.0-4975.g1889a165.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-17T17:26:38.469 INFO:teuthology.orchestra.run.smithi026.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 471 kB/s | 43 kB 00:00 2024-09-17T17:26:38.469 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T17:26:38.497 INFO:teuthology.orchestra.run.smithi026.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 448 kB/s | 11 kB 00:00 2024-09-17T17:26:38.512 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T17:26:38.514 INFO:teuthology.orchestra.run.smithi026.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-17T17:26:38.533 INFO:teuthology.orchestra.run.smithi026.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 1.4 MB/s | 23 kB 00:00 2024-09-17T17:26:38.547 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T17:26:38.556 INFO:teuthology.orchestra.run.smithi026.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-17T17:26:38.574 INFO:teuthology.orchestra.run.smithi026.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 465 kB/s | 19 kB 00:00 2024-09-17T17:26:38.590 INFO:teuthology.orchestra.run.smithi026.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 587 kB/s | 20 kB 00:00 2024-09-17T17:26:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T17:26:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-17T17:26:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T17:26:38.641 INFO:teuthology.orchestra.run.smithi026.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T17:26:38.683 INFO:teuthology.orchestra.run.smithi026.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-17T17:26:38.708 INFO:teuthology.orchestra.run.smithi026.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 244 kB/s | 16 kB 00:00 2024-09-17T17:26:38.711 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T17:26:38.734 INFO:teuthology.orchestra.run.smithi026.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-17T17:26:38.759 INFO:teuthology.orchestra.run.smithi026.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-17T17:26:38.784 INFO:teuthology.orchestra.run.smithi026.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-17T17:26:38.809 INFO:teuthology.orchestra.run.smithi026.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 810 kB/s | 20 kB 00:00 2024-09-17T17:26:38.832 INFO:teuthology.orchestra.run.smithi027.stdout:(6/37): ceph-osd-19.3.0-4975.g1889a165.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-17T17:26:38.835 INFO:teuthology.orchestra.run.smithi026.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T17:26:38.856 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T17:26:38.860 INFO:teuthology.orchestra.run.smithi026.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-17T17:26:38.864 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:38.864 INFO:teuthology.orchestra.run.smithi026.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-17T17:26:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T17:26:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T17:26:39.021 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:26:39.044 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T17:26:39.066 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:26:39.066 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:26:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T17:26:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T17:26:39.374 INFO:teuthology.orchestra.run.smithi027.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 251 kB/s | 253 kB 00:01 2024-09-17T17:26:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T17:26:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T17:26:39.471 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T17:26:39.512 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T17:26:39.557 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:39.558 INFO:teuthology.orchestra.run.smithi027.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 148 kB/s | 106 kB 00:00 2024-09-17T17:26:39.581 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:26:39.582 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:26:39.610 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T17:26:39.648 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T17:26:39.782 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T17:26:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 32/37 2024-09-17T17:26:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:40.019 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:40.019 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T17:26:40.019 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:40.020 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:40.083 INFO:teuthology.orchestra.run.smithi027.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 191 kB/s | 135 kB 00:00 2024-09-17T17:26:40.258 INFO:teuthology.orchestra.run.smithi027.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 178 kB/s | 125 kB 00:00 2024-09-17T17:26:40.392 INFO:teuthology.orchestra.run.smithi027.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-17T17:26:40.442 INFO:teuthology.orchestra.run.smithi027.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-17T17:26:40.484 INFO:teuthology.orchestra.run.smithi027.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 832 kB/s | 35 kB 00:00 2024-09-17T17:26:40.526 INFO:teuthology.orchestra.run.smithi027.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-17T17:26:40.568 INFO:teuthology.orchestra.run.smithi027.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-17T17:26:40.610 INFO:teuthology.orchestra.run.smithi027.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-17T17:26:40.669 INFO:teuthology.orchestra.run.smithi027.stdout:(17/37): python3-cryptography-36.0.1-4.el9.x86_ 568 kB/s | 1.2 MB 00:02 2024-09-17T17:26:40.719 INFO:teuthology.orchestra.run.smithi027.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-17T17:26:40.744 INFO:teuthology.orchestra.run.smithi027.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-09-17T17:26:40.770 INFO:teuthology.orchestra.run.smithi027.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 860 kB/s | 43 kB 00:00 2024-09-17T17:26:40.811 INFO:teuthology.orchestra.run.smithi027.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-17T17:26:40.837 INFO:teuthology.orchestra.run.smithi027.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 465 kB/s | 11 kB 00:00 2024-09-17T17:26:40.862 INFO:teuthology.orchestra.run.smithi027.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 710 kB/s | 18 kB 00:00 2024-09-17T17:26:40.887 INFO:teuthology.orchestra.run.smithi027.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 932 kB/s | 23 kB 00:00 2024-09-17T17:26:40.912 INFO:teuthology.orchestra.run.smithi027.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-17T17:26:40.929 INFO:teuthology.orchestra.run.smithi027.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 466 kB/s | 19 kB 00:00 2024-09-17T17:26:40.955 INFO:teuthology.orchestra.run.smithi027.stdout:(27/37): python3-urllib3-1.26.5-6.el9.noarch.rp 246 kB/s | 215 kB 00:00 2024-09-17T17:26:40.966 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:26:40.972 INFO:teuthology.orchestra.run.smithi027.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 333 kB/s | 20 kB 00:00 2024-09-17T17:26:41.006 INFO:teuthology.orchestra.run.smithi027.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 622 kB/s | 46 kB 00:00 2024-09-17T17:26:41.029 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T17:26:41.031 INFO:teuthology.orchestra.run.smithi027.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 714 kB/s | 16 kB 00:00 2024-09-17T17:26:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T17:26:41.064 INFO:teuthology.orchestra.run.smithi027.stdout:(31/37): python3-more-itertools-8.12.0-2.el9.no 723 kB/s | 79 kB 00:00 2024-09-17T17:26:41.090 INFO:teuthology.orchestra.run.smithi027.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-17T17:26:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T17:26:41.115 INFO:teuthology.orchestra.run.smithi027.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-17T17:26:41.140 INFO:teuthology.orchestra.run.smithi027.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-17T17:26:41.165 INFO:teuthology.orchestra.run.smithi027.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 800 kB/s | 20 kB 00:00 2024-09-17T17:26:41.166 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T17:26:41.191 INFO:teuthology.orchestra.run.smithi027.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T17:26:41.215 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T17:26:41.224 INFO:teuthology.orchestra.run.smithi027.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-17T17:26:41.230 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:41.230 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.2 MB/s | 31 MB 00:03 2024-09-17T17:26:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T17:26:41.344 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T17:26:41.390 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:26:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T17:26:41.434 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:26:41.434 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T17:26:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T17:26:41.531 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T17:26:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:41.566 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T17:26:41.588 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:41.588 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:41.588 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T17:26:41.588 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:41.588 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:41.588 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T17:26:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T17:26:41.683 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T17:26:41.717 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T17:26:41.767 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T17:26:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T17:26:41.830 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T17:26:41.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 1/4 2024-09-17T17:26:41.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2/4 2024-09-17T17:26:41.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T17:26:41.884 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T17:26:41.921 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:26:41.921 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:26:42.035 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T17:26:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:42.093 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:42.093 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:42.093 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T17:26:42.093 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:42.093 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:42.094 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:42.134 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T17:26:42.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T17:26:42.140 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:42.140 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:26:42.140 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:42.140 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:42.141 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T17:26:42.141 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T17:26:42.141 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:42.141 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:26:42.194 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T17:26:42.231 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T17:26:42.290 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T17:26:42.326 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:42.351 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:42.351 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:42.352 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T17:26:42.352 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:42.352 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:42.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:42.353 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T17:26:42.409 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:42.440 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph 2024-09-17T17:26:42.553 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T17:26:42.610 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T17:26:42.691 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T17:26:42.789 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T17:26:42.825 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T17:26:42.961 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T17:26:43.039 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:26:43.138 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 32/37 2024-09-17T17:26:43.168 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:43.182 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:26:43.184 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:43.184 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:26:43.184 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.3.0-4975.g1889a165.el9 ceph 6.6 k 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.3.0-4975.g1889a165.el9 ceph 2.4 M 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.3.0-4975.g1889a165.el9 ceph 1.5 M 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 291 k 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.3.0-4975.g1889a165.el9 ceph 4.8 M 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 17 M 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T17:26:43.185 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T17:26:43.186 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T17:26:43.187 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout:Install 37 Packages 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:43.188 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 31 M 2024-09-17T17:26:43.189 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 109 M 2024-09-17T17:26:43.189 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:26:43.196 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:43.196 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:43.196 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T17:26:43.196 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:43.196 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:43.196 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:26:43.528 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T17:26:43.581 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T17:26:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T17:26:43.693 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T17:26:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T17:26:43.897 INFO:teuthology.orchestra.run.smithi102.stdout:(1/37): ceph-19.3.0-4975.g1889a165.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-17T17:26:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T17:26:43.997 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T17:26:44.056 INFO:teuthology.orchestra.run.smithi102.stdout:(2/37): ceph-mgr-19.3.0-4975.g1889a165.el9.x86_ 5.3 MB/s | 1.5 MB 00:00 2024-09-17T17:26:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T17:26:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T17:26:44.139 INFO:teuthology.orchestra.run.smithi102.stdout:(3/37): ceph-mds-19.3.0-4975.g1889a165.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-17T17:26:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 1/37 2024-09-17T17:26:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2/37 2024-09-17T17:26:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 3/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 4/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 5/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 6/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T17:26:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T17:26:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T17:26:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T17:26:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T17:26:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T17:26:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T17:26:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T17:26:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T17:26:44.170 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T17:26:44.199 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T17:26:44.207 INFO:teuthology.orchestra.run.smithi102.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g1889 4.2 MB/s | 291 kB 00:00 2024-09-17T17:26:44.234 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T17:26:44.276 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T17:26:44.317 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T17:26:44.360 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T17:26:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T17:26:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T17:26:44.483 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T17:26:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T17:26:44.615 INFO:teuthology.orchestra.run.smithi102.stdout:(5/37): ceph-osd-19.3.0-4975.g1889a165.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-17T17:26:44.699 INFO:teuthology.orchestra.run.smithi102.stdout:(6/37): ceph-mon-19.3.0-4975.g1889a165.el9.x86_ 6.0 MB/s | 4.8 MB 00:00 2024-09-17T17:26:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T17:26:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:44.765 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:44.765 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T17:26:44.765 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:44.765 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:44.794 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T17:26:44.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T17:26:44.883 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T17:26:44.908 INFO:teuthology.orchestra.run.smithi102.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 361 kB/s | 253 kB 00:00 2024-09-17T17:26:44.942 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T17:26:45.006 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T17:26:45.024 INFO:teuthology.orchestra.run.smithi102.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-17T17:26:45.133 INFO:teuthology.orchestra.run.smithi102.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.1 MB/s | 125 kB 00:00 2024-09-17T17:26:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T17:26:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:45.189 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:26:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T17:26:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T17:26:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:45.192 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:45.200 INFO:teuthology.orchestra.run.smithi102.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 213 kB/s | 106 kB 00:00 2024-09-17T17:26:45.203 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T17:26:45.245 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:45.251 INFO:teuthology.orchestra.run.smithi102.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-09-17T17:26:45.255 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T17:26:45.273 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:45.274 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:45.274 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T17:26:45.274 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:45.274 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:45.274 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:45.328 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T17:26:45.409 INFO:teuthology.orchestra.run.smithi102.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 901 kB/s | 188 kB 00:00 2024-09-17T17:26:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T17:26:45.443 INFO:teuthology.orchestra.run.smithi102.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 896 kB/s | 172 kB 00:00 2024-09-17T17:26:45.468 INFO:teuthology.orchestra.run.smithi102.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 594 kB/s | 35 kB 00:00 2024-09-17T17:26:45.470 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T17:26:45.493 INFO:teuthology.orchestra.run.smithi102.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-17T17:26:45.504 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:45.519 INFO:teuthology.orchestra.run.smithi102.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T17:26:45.529 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:45.529 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:45.529 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T17:26:45.530 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:45.530 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:45.530 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:45.544 INFO:teuthology.orchestra.run.smithi102.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-17T17:26:45.578 INFO:teuthology.orchestra.run.smithi102.stdout:(18/37): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2024-09-17T17:26:45.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-base 2024-09-17T17:26:45.597 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T17:26:45.620 INFO:teuthology.orchestra.run.smithi102.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 221 kB/s | 22 kB 00:00 2024-09-17T17:26:45.653 INFO:teuthology.orchestra.run.smithi102.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 640 kB/s | 43 kB 00:00 2024-09-17T17:26:45.680 INFO:teuthology.orchestra.run.smithi102.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-17T17:26:45.704 INFO:teuthology.orchestra.run.smithi102.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 429 kB/s | 11 kB 00:00 2024-09-17T17:26:45.729 INFO:teuthology.orchestra.run.smithi102.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-17T17:26:45.754 INFO:teuthology.orchestra.run.smithi102.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-17T17:26:45.775 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 32/37 2024-09-17T17:26:45.779 INFO:teuthology.orchestra.run.smithi102.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-17T17:26:45.797 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:45.805 INFO:teuthology.orchestra.run.smithi102.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-17T17:26:45.824 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T17:26:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:45.825 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:45.825 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:45.830 INFO:teuthology.orchestra.run.smithi102.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-17T17:26:45.855 INFO:teuthology.orchestra.run.smithi102.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-17T17:26:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-17T17:26:45.910 INFO:teuthology.orchestra.run.smithi102.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 994 kB/s | 79 kB 00:00 2024-09-17T17:26:45.935 INFO:teuthology.orchestra.run.smithi102.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-17T17:26:45.960 INFO:teuthology.orchestra.run.smithi102.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T17:26:45.985 INFO:teuthology.orchestra.run.smithi102.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-17T17:26:46.010 INFO:teuthology.orchestra.run.smithi102.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-17T17:26:46.036 INFO:teuthology.orchestra.run.smithi102.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-17T17:26:46.061 INFO:teuthology.orchestra.run.smithi102.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-17T17:26:46.088 INFO:teuthology.orchestra.run.smithi102.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-17T17:26:46.090 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:46.090 INFO:teuthology.orchestra.run.smithi102.stdout:Total 11 MB/s | 31 MB 00:02 2024-09-17T17:26:46.201 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:46.239 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:26:46.279 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:26:46.279 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:26:46.293 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:46.337 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:46.338 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:26:46.338 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:46.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install cephadm 2024-09-17T17:26:46.753 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:26:46.753 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:26:47.005 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:47.141 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:47.142 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:26:47.142 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 784 k 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 784 k 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 791 k 2024-09-17T17:26:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 1/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 3/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 4/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 5/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 6/37 2024-09-17T17:26:47.337 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T17:26:47.338 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T17:26:47.339 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T17:26:47.339 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T17:26:47.339 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T17:26:47.339 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T17:26:47.339 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T17:26:47.339 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T17:26:47.340 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T17:26:47.340 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T17:26:47.340 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T17:26:47.340 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T17:26:47.340 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T17:26:47.341 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T17:26:47.341 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T17:26:47.341 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T17:26:47.341 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T17:26:47.341 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T17:26:47.341 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T17:26:47.364 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:47.393 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:47.393 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T17:26:47.393 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:47.393 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm-19.3.0-4975.g1889a165.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-17T17:26:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.0 MB/s | 784 kB 00:00 2024-09-17T17:26:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:26:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:26:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:26:47.413 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:26:47.413 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:26:47.489 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:26:47.873 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:47.898 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:47.898 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:47.899 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T17:26:47.899 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:47.899 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:47.899 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:48.116 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:26:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:48.158 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:48.158 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:48.158 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T17:26:48.158 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:48.158 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:48.158 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:48.164 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T17:26:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T17:26:48.223 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T17:26:48.313 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T17:26:48.362 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T17:26:48.398 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T17:26:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:48.491 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:48.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T17:26:48.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T17:26:48.501 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:48.502 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:48.569 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T17:26:48.610 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T17:26:48.644 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T17:26:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T17:26:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T17:26:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:48.719 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:26:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:48.719 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T17:26:48.791 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T17:26:48.826 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T17:26:48.852 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-base 2024-09-17T17:26:48.860 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T17:26:48.909 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T17:26:48.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T17:26:48.945 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T17:26:49.014 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T17:26:49.152 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T17:26:49.233 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T17:26:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T17:26:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T17:26:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T17:26:49.439 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T17:26:49.455 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:49.547 INFO:teuthology.orchestra.run.smithi026.stdout:Package ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:49.547 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:49.592 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:49.593 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:26:49.593 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:49.632 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T17:26:49.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install cephadm 2024-09-17T17:26:49.680 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T17:26:49.687 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:49.687 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T17:26:49.687 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g1889a165.el9 ceph 145 k 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:26:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:49.689 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 145 k 2024-09-17T17:26:49.689 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 432 k 2024-09-17T17:26:49.689 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:26:49.759 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T17:26:49.847 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T17:26:49.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T17:26:49.890 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-immutable-object-cache-19.3.0-4975.g1889a1 718 kB/s | 145 kB 00:00 2024-09-17T17:26:49.891 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:49.891 INFO:teuthology.orchestra.run.smithi007.stdout:Total 712 kB/s | 145 kB 00:00 2024-09-17T17:26:49.891 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:26:49.900 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:26:49.900 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 1/37 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2/37 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 3/37 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 4/37 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 5/37 2024-09-17T17:26:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 6/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T17:26:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T17:26:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T17:26:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T17:26:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T17:26:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T17:26:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T17:26:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T17:26:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T17:26:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T17:26:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T17:26:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T17:26:49.960 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:26:49.960 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:26:50.009 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T17:26:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:26:50.169 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 32/37 2024-09-17T17:26:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:50.198 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:50.218 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:50.218 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T17:26:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:50.224 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T17:26:50.224 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:50.224 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T17:26:50.225 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:50.225 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T17:26:50.225 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:50.281 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:50.417 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 784 k 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:50.418 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:26:50.419 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:50.419 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 784 k 2024-09-17T17:26:50.419 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 791 k 2024-09-17T17:26:50.419 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:26:50.702 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm-19.3.0-4975.g1889a165.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-17T17:26:50.703 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:50.703 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-17T17:26:50.703 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:26:50.708 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:26:50.709 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:26:50.715 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:26:50.715 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:26:50.788 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:26:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T17:26:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:50.971 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:26:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T17:26:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T17:26:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T17:26:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T17:26:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:50.975 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:51.172 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:26:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:51.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:51.172 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:51.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-base 2024-09-17T17:26:51.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr 2024-09-17T17:26:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:51.691 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:51.718 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T17:26:51.718 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:51.718 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T17:26:51.718 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:51.719 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T17:26:51.719 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:26:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:51.984 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:52.019 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:03 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:52.077 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:52.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T17:26:52.111 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:52.121 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:52.122 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:26:52.122 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:52.156 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:52.157 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:26:52.157 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:26:52.175 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:52.197 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install cephadm 2024-09-17T17:26:52.198 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T17:26:52.198 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:52.198 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T17:26:52.198 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:52.198 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T17:26:52.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:52.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T17:26:52.425 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:52.449 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T17:26:52.449 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:52.449 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T17:26:52.449 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:52.449 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T17:26:52.449 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:52.514 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:52.686 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:52.803 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:52.826 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:52.826 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g1889a165.el9 ceph 145 k 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 145 k 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 432 k 2024-09-17T17:26:52.827 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:26:52.828 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:03 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:26:52.940 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:52.941 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:52.941 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:26:52.941 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:52.941 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:26:52.941 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 784 k 2024-09-17T17:26:52.941 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 784 k 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 791 k 2024-09-17T17:26:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:26:52.966 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:26:52.967 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:26:52.967 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:52.967 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:26:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 14 M 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 34 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 16 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T17:26:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:26:52.969 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 19 M 2024-09-17T17:26:52.970 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 159 M 2024-09-17T17:26:52.970 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:26:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-immutable-object-cache-19.3.0-4975.g1889a1 682 kB/s | 145 kB 00:00 2024-09-17T17:26:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:Total 676 kB/s | 145 kB 00:00 2024-09-17T17:26:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:26:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:26:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:26:53.110 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:26:53.110 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:26:53.214 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm-19.3.0-4975.g1889a165.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-17T17:26:53.215 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:53.215 INFO:teuthology.orchestra.run.smithi027.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-17T17:26:53.215 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:26:53.221 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:26:53.221 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:26:53.227 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:26:53.227 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:26:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:26:53.295 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4975.g18 206 kB/s | 34 kB 00:00 2024-09-17T17:26:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:26:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:53.320 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4975.g188 86 kB/s | 16 kB 00:00 2024-09-17T17:26:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:53.343 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:53.343 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T17:26:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:53.702 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:53.813 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4975.g1889a16 21 MB/s | 14 MB 00:00 2024-09-17T17:26:53.979 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.1 MB/s | 863 kB 00:00 2024-09-17T17:26:54.030 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-17T17:26:54.055 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-09-17T17:26:54.122 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-17T17:26:54.147 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-17T17:26:54.173 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-17T17:26:54.198 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-17T17:26:54.221 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 1/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 3/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 4/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 5/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 6/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T17:26:54.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T17:26:54.298 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T17:26:54.299 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T17:26:54.299 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T17:26:54.299 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T17:26:54.299 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T17:26:54.299 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T17:26:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:54.300 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:26:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:54.300 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:54.300 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:54.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T17:26:54.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T17:26:54.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T17:26:54.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T17:26:54.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T17:26:54.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T17:26:54.302 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T17:26:54.302 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T17:26:54.302 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T17:26:54.323 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-protobuf-3.14.0-13.el9.noarch. 269 kB/s | 269 kB 00:01 2024-09-17T17:26:54.468 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:54.468 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:54.468 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:26:54.468 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:54.468 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:54.468 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:54.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr 2024-09-17T17:26:54.690 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T17:26:55.122 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:55.214 INFO:teuthology.orchestra.run.smithi026.stdout:Package ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 531 kB/s | 1.0 MB 00:01 2024-09-17T17:26:55.245 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:55.245 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.5 MB/s | 19 MB 00:02 2024-09-17T17:26:55.259 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:55.260 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:26:55.260 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:26:55.300 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:26:55.302 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:55.318 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T17:26:55.318 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:55.318 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T17:26:55.319 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T17:26:55.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T17:26:55.321 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T17:26:55.322 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:55.322 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:26:55.322 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:26:55.322 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:26:55.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T17:26:55.439 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:55.440 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:55.440 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-17T17:26:55.440 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:55.440 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:26:55.440 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g1889a165.el9 ceph 145 k 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 145 k 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 432 k 2024-09-17T17:26:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:26:55.624 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:26:55.625 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:26:55.663 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-immutable-object-cache-19.3.0-4975.g1889a1 653 kB/s | 145 kB 00:00 2024-09-17T17:26:55.663 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:55.664 INFO:teuthology.orchestra.run.smithi027.stdout:Total 649 kB/s | 145 kB 00:00 2024-09-17T17:26:55.664 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:26:55.672 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:26:55.672 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:26:55.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base 2024-09-17T17:26:55.732 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:26:55.733 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:26:55.882 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:26:55.942 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:26:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:55.982 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:26:55.982 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T17:26:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:56.084 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:26:56.085 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:26:56.085 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:56.085 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 14 M 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 34 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 16 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T17:26:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout:Install 12 Packages 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 19 M 2024-09-17T17:26:56.087 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 159 M 2024-09-17T17:26:56.088 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:26:56.235 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:26:56.293 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:26:56.308 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T17:26:56.352 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T17:26:56.382 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T17:26:56.425 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:26:56.425 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:26:56.426 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:26:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T17:26:56.490 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install cephadm 2024-09-17T17:26:56.543 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T17:26:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T17:26:56.752 INFO:teuthology.orchestra.run.smithi026.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4975.g188 104 kB/s | 16 kB 00:00 2024-09-17T17:26:56.769 INFO:teuthology.orchestra.run.smithi026.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4975.g18 195 kB/s | 34 kB 00:00 2024-09-17T17:26:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T17:26:56.851 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T17:26:56.887 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:26:56.918 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:26:56.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:56.918 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:26:56.918 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:26:56.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:56.918 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:56.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 10/12 2024-09-17T17:26:56.994 INFO:teuthology.orchestra.run.smithi026.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-17T17:26:57.067 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:26:57.116 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr 2024-09-17T17:26:57.197 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 784 k 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:57.198 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 784 k 2024-09-17T17:26:57.199 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 791 k 2024-09-17T17:26:57.199 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:26:57.228 INFO:teuthology.orchestra.run.smithi026.stdout:(4/12): ceph-mgr-dashboard-19.3.0-4975.g1889a16 22 MB/s | 14 MB 00:00 2024-09-17T17:26:57.261 INFO:teuthology.orchestra.run.smithi026.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-09-17T17:26:57.329 INFO:teuthology.orchestra.run.smithi026.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.4 MB/s | 548 kB 00:00 2024-09-17T17:26:57.345 INFO:teuthology.orchestra.run.smithi026.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 255 kB/s | 19 kB 00:00 2024-09-17T17:26:57.370 INFO:teuthology.orchestra.run.smithi026.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-17T17:26:57.387 INFO:teuthology.orchestra.run.smithi026.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.8 MB/s | 31 kB 00:00 2024-09-17T17:26:57.421 INFO:teuthology.orchestra.run.smithi026.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.0 MB/s | 863 kB 00:00 2024-09-17T17:26:57.446 INFO:teuthology.orchestra.run.smithi026.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-17T17:26:57.466 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm-19.3.0-4975.g1889a165.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-17T17:26:57.466 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:57.467 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-17T17:26:57.467 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:26:57.472 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:26:57.472 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:26:57.478 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:26:57.479 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:26:57.496 INFO:teuthology.orchestra.run.smithi026.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-17T17:26:57.500 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:57.500 INFO:teuthology.orchestra.run.smithi026.stdout:Total 14 MB/s | 19 MB 00:01 2024-09-17T17:26:57.548 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:26:57.550 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:26:57.570 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:26:57.570 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:26:57.725 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:57.816 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:26:57.860 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:26:57.861 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:57.861 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:26:57.862 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:26:57.862 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:26:57.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T17:26:57.945 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:58.036 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:58.422 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:58.464 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:26:58.533 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:26:58.547 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T17:26:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T17:26:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T17:26:58.671 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 14 M 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T17:26:58.673 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 34 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 16 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:26:58.674 INFO:teuthology.orchestra.run.smithi027.stdout:Install 12 Packages 2024-09-17T17:26:58.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:26:58.675 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 19 M 2024-09-17T17:26:58.675 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 159 M 2024-09-17T17:26:58.675 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:26:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T17:26:58.688 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:26:58.688 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:58.688 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:26:58.688 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:26:58.688 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:58.688 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:26:58.788 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T17:26:58.872 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T17:26:58.918 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T17:26:59.019 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T17:26:59.059 INFO:teuthology.orchestra.run.smithi027.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4975.g18 231 kB/s | 34 kB 00:00 2024-09-17T17:26:59.066 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T17:26:59.084 INFO:teuthology.orchestra.run.smithi027.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4975.g188 95 kB/s | 16 kB 00:00 2024-09-17T17:26:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:26:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 10/12 2024-09-17T17:26:59.334 INFO:teuthology.orchestra.run.smithi027.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.1 MB/s | 269 kB 00:00 2024-09-17T17:26:59.401 INFO:teuthology.orchestra.run.smithi027.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-09-17T17:26:59.461 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:26:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.8 MB/s | 548 kB 00:00 2024-09-17T17:26:59.518 INFO:teuthology.orchestra.run.smithi027.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-17T17:26:59.569 INFO:teuthology.orchestra.run.smithi027.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.6 MB/s | 863 kB 00:00 2024-09-17T17:26:59.598 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g1889a165.el9 ceph 145 k 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:26:59.599 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:26:59.600 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:26:59.600 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:26:59.600 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 145 k 2024-09-17T17:26:59.600 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 432 k 2024-09-17T17:26:59.600 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:26:59.627 INFO:teuthology.orchestra.run.smithi027.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 19 MB/s | 2.0 MB 00:00 2024-09-17T17:26:59.661 INFO:teuthology.orchestra.run.smithi027.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 986 kB/s | 31 kB 00:00 2024-09-17T17:26:59.694 INFO:teuthology.orchestra.run.smithi027.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.6 MB/s | 188 kB 00:00 2024-09-17T17:26:59.719 INFO:teuthology.orchestra.run.smithi027.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 959 kB/s | 144 kB 00:00 2024-09-17T17:26:59.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-immutable-object-cache-19.3.0-4975.g1889a1 620 kB/s | 145 kB 00:00 2024-09-17T17:26:59.833 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:59.833 INFO:teuthology.orchestra.run.smithi102.stdout:Total 617 kB/s | 145 kB 00:00 2024-09-17T17:26:59.834 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:26:59.841 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:26:59.841 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:26:59.898 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:26:59.899 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:26:59.911 INFO:teuthology.orchestra.run.smithi027.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4975.g1889a1 14 MB/s | 14 MB 00:01 2024-09-17T17:26:59.916 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:26:59.916 INFO:teuthology.orchestra.run.smithi027.stdout:Total 16 MB/s | 19 MB 00:01 2024-09-17T17:26:59.967 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:26:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:26:59.988 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:27:00.126 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:27:00.126 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:00.126 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T17:27:00.126 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:00.275 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:00.276 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 11/12 2024-09-17T17:27:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T17:27:00.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T17:27:01.039 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T17:27:01.040 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:01.040 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:01.040 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:01.040 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:01.040 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:01.046 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T17:27:01.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T17:27:01.220 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T17:27:01.245 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr 2024-09-17T17:27:01.365 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T17:27:01.466 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T17:27:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T17:27:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:27:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 10/12 2024-09-17T17:27:01.841 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:01.928 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:01.970 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:01.971 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:01.971 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:02.039 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T17:27:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 1/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 2/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 3/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T17:27:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T17:27:02.535 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 11/12 2024-09-17T17:27:02.588 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:02.615 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T17:27:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:02.731 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 14 M 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 34 k 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 16 k 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T17:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout:Install 12 Packages 2024-09-17T17:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 19 M 2024-09-17T17:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 159 M 2024-09-17T17:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:02.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T17:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4975.g18 118 kB/s | 34 kB 00:00 2024-09-17T17:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4975.g188 52 kB/s | 16 kB 00:00 2024-09-17T17:27:03.592 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:14 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:03.729 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:03.729 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 7.4 M 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T17:27:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-17T17:27:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:03.732 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 38 M 2024-09-17T17:27:03.732 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 212 M 2024-09-17T17:27:03.732 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:04.215 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-17T17:27:04.273 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.4 MB/s | 801 kB 00:00 2024-09-17T17:27:04.287 INFO:teuthology.orchestra.run.smithi102.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 323 kB/s | 269 kB 00:00 2024-09-17T17:27:04.365 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 218 kB/s | 33 kB 00:00 2024-09-17T17:27:04.424 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-09-17T17:27:04.429 INFO:teuthology.orchestra.run.smithi102.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-09-17T17:27:04.499 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 514 kB/s | 38 kB 00:00 2024-09-17T17:27:04.513 INFO:teuthology.orchestra.run.smithi102.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.7 MB/s | 863 kB 00:00 2024-09-17T17:27:04.563 INFO:teuthology.orchestra.run.smithi102.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.0 MB/s | 548 kB 00:00 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 1/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 2/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 3/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T17:27:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T17:27:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T17:27:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:27:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T17:27:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T17:27:04.588 INFO:teuthology.orchestra.run.smithi102.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-09-17T17:27:04.600 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-09-17T17:27:04.647 INFO:teuthology.orchestra.run.smithi102.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 24 MB/s | 2.0 MB 00:00 2024-09-17T17:27:04.650 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-17T17:27:04.673 INFO:teuthology.orchestra.run.smithi102.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.7 MB/s | 144 kB 00:00 2024-09-17T17:27:04.698 INFO:teuthology.orchestra.run.smithi102.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 611 kB/s | 31 kB 00:00 2024-09-17T17:27:04.732 INFO:teuthology.orchestra.run.smithi102.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-17T17:27:04.792 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-17T17:27:04.884 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-17T17:27:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T17:27:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:04.897 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T17:27:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:04.899 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:04.927 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-09-17T17:27:04.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 11/12 2024-09-17T17:27:05.044 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:05.082 INFO:teuthology.orchestra.run.smithi102.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4975.g1889a1 7.3 MB/s | 14 MB 00:01 2024-09-17T17:27:05.086 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:05.086 INFO:teuthology.orchestra.run.smithi102.stdout:Total 8.2 MB/s | 19 MB 00:02 2024-09-17T17:27:05.136 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:05.155 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:05.156 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:05.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T17:27:05.243 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-09-17T17:27:05.433 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:05.433 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:05.760 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:14 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:05.777 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): ceph-mgr-diskprediction-local-19.3.0-4 4.3 MB/s | 7.4 MB 00:01 2024-09-17T17:27:05.778 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:05.778 INFO:teuthology.orchestra.run.smithi007.stdout:Total 19 MB/s | 38 MB 00:02 2024-09-17T17:27:05.897 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:05.898 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-09-17T17:27:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:27:05.898 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-09-17T17:27:05.898 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 7.4 M 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T17:27:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:05.900 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:05.900 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-09-17T17:27:05.900 INFO:teuthology.orchestra.run.smithi026.stdout:Install 12 Packages 2024-09-17T17:27:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:05.900 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 38 M 2024-09-17T17:27:05.900 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 212 M 2024-09-17T17:27:05.901 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:06.065 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:06.146 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:06.157 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T17:27:06.170 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:06.170 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:06.193 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T17:27:06.239 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T17:27:06.269 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T17:27:06.374 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T17:27:06.504 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T17:27:06.597 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T17:27:06.631 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:06.631 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:06.643 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T17:27:06.669 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:27:06.701 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 10/12 2024-09-17T17:27:06.820 INFO:teuthology.orchestra.run.smithi026.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 327 kB/s | 191 kB 00:00 2024-09-17T17:27:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 1/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 2/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 3/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T17:27:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T17:27:06.962 INFO:teuthology.orchestra.run.smithi026.stdout:(2/12): ceph-mgr-diskprediction-local-19.3.0-49 10 MB/s | 7.4 MB 00:00 2024-09-17T17:27:06.987 INFO:teuthology.orchestra.run.smithi026.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-09-17T17:27:07.046 INFO:teuthology.orchestra.run.smithi026.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 313 kB/s | 18 kB 00:00 2024-09-17T17:27:07.138 INFO:teuthology.orchestra.run.smithi026.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 421 kB/s | 38 kB 00:00 2024-09-17T17:27:07.173 INFO:teuthology.orchestra.run.smithi026.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 857 kB/s | 801 kB 00:00 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:07.234 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:07.446 INFO:teuthology.orchestra.run.smithi026.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 890 kB/s | 245 kB 00:00 2024-09-17T17:27:07.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T17:27:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T17:27:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:07.688 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:07.727 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T17:27:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T17:27:07.942 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T17:27:07.972 INFO:teuthology.orchestra.run.smithi026.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.0 MB/s | 3.0 MB 00:01 2024-09-17T17:27:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T17:27:08.072 INFO:teuthology.orchestra.run.smithi026.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.5 MB/s | 250 kB 00:00 2024-09-17T17:27:08.079 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:15 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:08.219 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:08.220 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-17T17:27:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:27:08.220 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-17T17:27:08.220 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 7.4 M 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T17:27:08.221 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T17:27:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:08.222 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:08.222 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-17T17:27:08.222 INFO:teuthology.orchestra.run.smithi027.stdout:Install 12 Packages 2024-09-17T17:27:08.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:08.223 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 38 M 2024-09-17T17:27:08.223 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 212 M 2024-09-17T17:27:08.223 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:08.447 INFO:teuthology.orchestra.run.smithi026.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.8 MB/s | 4.9 MB 00:01 2024-09-17T17:27:08.718 INFO:teuthology.orchestra.run.smithi027.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-17T17:27:08.720 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T17:27:08.769 INFO:teuthology.orchestra.run.smithi027.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-17T17:27:08.794 INFO:teuthology.orchestra.run.smithi027.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-09-17T17:27:08.839 INFO:teuthology.orchestra.run.smithi026.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-09-17T17:27:08.844 INFO:teuthology.orchestra.run.smithi027.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-09-17T17:27:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T17:27:08.886 INFO:teuthology.orchestra.run.smithi027.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 925 kB/s | 38 kB 00:00 2024-09-17T17:27:09.111 INFO:teuthology.orchestra.run.smithi027.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-49 13 MB/s | 7.4 MB 00:00 2024-09-17T17:27:09.245 INFO:teuthology.orchestra.run.smithi027.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-17T17:27:09.587 INFO:teuthology.orchestra.run.smithi027.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.7 MB/s | 3.0 MB 00:00 2024-09-17T17:27:09.687 INFO:teuthology.orchestra.run.smithi027.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.5 MB/s | 250 kB 00:00 2024-09-17T17:27:09.885 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 11/12 2024-09-17T17:27:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:10.172 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:10.287 INFO:teuthology.orchestra.run.smithi027.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.5 MB/s | 4.9 MB 00:01 2024-09-17T17:27:10.389 INFO:teuthology.orchestra.run.smithi026.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-09-17T17:27:10.390 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:10.390 INFO:teuthology.orchestra.run.smithi026.stdout:Total 8.5 MB/s | 38 MB 00:04 2024-09-17T17:27:10.646 INFO:teuthology.orchestra.run.smithi027.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-09-17T17:27:10.762 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:10.789 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:10.789 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:11.029 INFO:teuthology.orchestra.run.smithi027.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-17T17:27:11.030 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:11.031 INFO:teuthology.orchestra.run.smithi027.stdout:Total 14 MB/s | 38 MB 00:02 2024-09-17T17:27:11.244 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:11.244 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:11.259 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T17:27:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:11.397 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:11.425 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:11.425 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:11.785 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T17:27:11.785 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 1/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 2/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 3/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T17:27:11.786 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T17:27:11.890 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:11.891 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:12.046 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:12.117 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T17:27:12.136 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T17:27:12.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:12.136 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T17:27:12.136 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:12.136 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:12.154 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:12.250 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T17:27:12.383 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T17:27:12.482 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T17:27:12.512 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T17:27:12.592 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T17:27:12.798 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:12.879 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T17:27:12.929 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:08 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:13.044 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T17:27:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-17T17:27:13.101 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:27:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-17T17:27:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 7.4 M 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T17:27:13.102 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:Install 12 Packages 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 38 M 2024-09-17T17:27:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 212 M 2024-09-17T17:27:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:13.271 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T17:27:13.279 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T17:27:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-17T17:27:13.294 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T17:27:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-17T17:27:13.374 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T17:27:13.406 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T17:27:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-09-17T17:27:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-09-17T17:27:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 774 kB/s | 38 kB 00:00 2024-09-17T17:27:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-49 15 MB/s | 7.4 MB 00:00 2024-09-17T17:27:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-17T17:27:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 1/12 2024-09-17T17:27:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T17:27:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T17:27:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T17:27:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T17:27:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T17:27:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T17:27:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T17:27:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T17:27:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.5 MB/s | 3.0 MB 00:00 2024-09-17T17:27:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T17:27:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T17:27:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:14.388 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.4 MB/s | 4.9 MB 00:01 2024-09-17T17:27:14.730 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:14.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-rook 2024-09-17T17:27:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:26 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-17T17:27:15.543 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:15.544 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:27:15.544 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 49 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T17:27:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-09-17T17:27:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:15.547 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.2 M 2024-09-17T17:27:15.547 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 26 M 2024-09-17T17:27:15.547 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:15.827 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T17:27:15.853 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:16.094 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 180 kB/s | 19 kB 00:00 2024-09-17T17:27:16.119 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 194 kB/s | 26 kB 00:00 2024-09-17T17:27:16.144 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): ceph-mgr-rook-19.3.0-4975.g1889a165.el9 314 kB/s | 49 kB 00:00 2024-09-17T17:27:16.195 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-17T17:27:16.228 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-17T17:27:16.253 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-09-17T17:27:16.296 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 279 kB 00:00 2024-09-17T17:27:16.322 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-17T17:27:16.358 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-17T17:27:16.389 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-17T17:27:16.431 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.6 MB/s | 220 kB 00:00 2024-09-17T17:27:16.464 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-09-17T17:27:16.492 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-17T17:27:16.496 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:16.496 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-17T17:27:16.559 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:16.576 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:16.576 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:16.670 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T17:27:16.676 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:16.694 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-17T17:27:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:Total 11 MB/s | 38 MB 00:03 2024-09-17T17:27:17.061 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:17.088 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:17.088 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:17.321 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T17:27:17.514 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:17.514 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:17.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T17:27:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T17:27:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T17:27:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T17:27:17.924 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T17:27:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T17:27:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T17:27:18.290 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:18.353 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T17:27:18.398 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:18.432 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:18.432 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 1/12 2024-09-17T17:27:18.432 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T17:27:18.500 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:18.533 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T17:27:18.717 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T17:27:18.740 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T17:27:18.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T17:27:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T17:27:18.871 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T17:27:18.872 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T17:27:18.873 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:18.873 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:18.873 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T17:27:18.873 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:18.873 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:18.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T17:27:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T17:27:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T17:27:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:19.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-rook 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 1/12 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T17:27:19.328 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T17:27:19.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T17:27:19.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T17:27:19.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T17:27:19.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:19.329 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T17:27:19.473 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T17:27:19.587 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 1/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T17:27:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T17:27:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T17:27:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T17:27:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T17:27:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T17:27:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:19.750 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:19.855 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:19.995 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:19.996 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 49 k 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T17:27:19.997 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:Install 13 Packages 2024-09-17T17:27:19.998 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 2.2 M 2024-09-17T17:27:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 26 M 2024-09-17T17:27:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T17:27:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:20.077 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:20.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-rook 2024-09-17T17:27:20.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T17:27:20.620 INFO:teuthology.orchestra.run.smithi026.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 284 kB/s | 26 kB 00:00 2024-09-17T17:27:20.637 INFO:teuthology.orchestra.run.smithi026.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 179 kB/s | 19 kB 00:00 2024-09-17T17:27:20.701 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:27 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:20.712 INFO:teuthology.orchestra.run.smithi026.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-17T17:27:20.737 INFO:teuthology.orchestra.run.smithi026.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-17T17:27:20.763 INFO:teuthology.orchestra.run.smithi026.stdout:(5/13): ceph-mgr-rook-19.3.0-4975.g1889a165.el9 212 kB/s | 49 kB 00:00 2024-09-17T17:27:20.788 INFO:teuthology.orchestra.run.smithi026.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T17:27:20.813 INFO:teuthology.orchestra.run.smithi026.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-17T17:27:20.839 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 49 k 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T17:27:20.841 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout:Install 13 Packages 2024-09-17T17:27:20.842 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:20.843 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:20.843 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 2.2 M 2024-09-17T17:27:20.843 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 26 M 2024-09-17T17:27:20.843 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:20.863 INFO:teuthology.orchestra.run.smithi026.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 321 kB/s | 32 kB 00:00 2024-09-17T17:27:20.889 INFO:teuthology.orchestra.run.smithi026.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 141 kB/s | 14 kB 00:00 2024-09-17T17:27:20.922 INFO:teuthology.orchestra.run.smithi026.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-17T17:27:20.948 INFO:teuthology.orchestra.run.smithi026.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-17T17:27:20.954 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:20.973 INFO:teuthology.orchestra.run.smithi026.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-17T17:27:21.006 INFO:teuthology.orchestra.run.smithi026.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-17T17:27:21.009 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:21.010 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.2 MB/s | 2.2 MB 00:01 2024-09-17T17:27:21.028 INFO:teuthology.orchestra.run.smithi027.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 209 kB/s | 19 kB 00:00 2024-09-17T17:27:21.053 INFO:teuthology.orchestra.run.smithi027.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 219 kB/s | 26 kB 00:00 2024-09-17T17:27:21.080 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:21.095 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 154 k 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T17:27:21.096 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T17:27:21.097 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T17:27:21.097 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T17:27:21.097 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T17:27:21.097 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:21.097 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:21.097 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:21.100 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-09-17T17:27:21.100 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:21.101 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:21.101 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:21.101 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 7.0 M 2024-09-17T17:27:21.101 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 33 M 2024-09-17T17:27:21.101 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:21.120 INFO:teuthology.orchestra.run.smithi027.stdout:(3/13): ceph-mgr-rook-19.3.0-4975.g1889a165.el9 268 kB/s | 49 kB 00:00 2024-09-17T17:27:21.145 INFO:teuthology.orchestra.run.smithi027.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-17T17:27:21.170 INFO:teuthology.orchestra.run.smithi027.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-17T17:27:21.194 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:21.195 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:21.195 INFO:teuthology.orchestra.run.smithi027.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T17:27:21.237 INFO:teuthology.orchestra.run.smithi027.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-17T17:27:21.263 INFO:teuthology.orchestra.run.smithi027.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 210 kB/s | 14 kB 00:00 2024-09-17T17:27:21.280 INFO:teuthology.orchestra.run.smithi027.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.7 MB/s | 279 kB 00:00 2024-09-17T17:27:21.314 INFO:teuthology.orchestra.run.smithi027.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-17T17:27:21.355 INFO:teuthology.orchestra.run.smithi027.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.2 MB/s | 90 kB 00:00 2024-09-17T17:27:21.381 INFO:teuthology.orchestra.run.smithi027.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-09-17T17:27:21.414 INFO:teuthology.orchestra.run.smithi027.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-17T17:27:21.417 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:21.417 INFO:teuthology.orchestra.run.smithi027.stdout:Total 3.9 MB/s | 2.2 MB 00:00 2024-09-17T17:27:21.485 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:21.501 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:21.501 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:21.598 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:21.600 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:21.608 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-17T17:27:21.683 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4975.g1889a165.e 621 kB/s | 154 kB 00:00 2024-09-17T17:27:21.710 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:21.725 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.6 MB/s | 548 kB 00:00 2024-09-17T17:27:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.1 MB/s | 74 kB 00:00 2024-09-17T17:27:21.776 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 625 kB/s | 58 kB 00:00 2024-09-17T17:27:21.860 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-17T17:27:21.862 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:21.862 INFO:teuthology.orchestra.run.smithi007.stdout:Total 9.2 MB/s | 7.0 MB 00:00 2024-09-17T17:27:21.890 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T17:27:21.894 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T17:27:21.924 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:21.936 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:21.948 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:21.948 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:21.952 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T17:27:22.008 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T17:27:22.042 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T17:27:22.055 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:22.056 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:22.085 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T17:27:22.192 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T17:27:22.245 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:22.303 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T17:27:22.354 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T17:27:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T17:27:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T17:27:22.573 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T17:27:22.608 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T17:27:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T17:27:22.659 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T17:27:22.713 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T17:27:22.765 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T17:27:22.876 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T17:27:22.936 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T17:27:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T17:27:23.200 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T17:27:23.233 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T17:27:23.265 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T17:27:23.282 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T17:27:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:23.368 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T17:27:23.444 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T17:27:23.474 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T17:27:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T17:27:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T17:27:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T17:27:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 1/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T17:27:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T17:27:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T17:27:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T17:27:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T17:27:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T17:27:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T17:27:24.277 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:24.278 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/6 2024-09-17T17:27:24.278 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T17:27:24.278 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T17:27:24.278 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T17:27:24.278 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T17:27:24.470 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:24.471 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T17:27:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:24.566 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T17:27:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T17:27:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T17:27:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T17:27:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T17:27:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:24.567 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 1/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T17:27:24.648 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T17:27:24.648 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 1/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T17:27:24.649 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T17:27:24.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T17:27:24.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-fuse 2024-09-17T17:27:24.934 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T17:27:24.935 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T17:27:24.936 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T17:27:24.936 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T17:27:24.936 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T17:27:24.936 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T17:27:24.936 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:24.936 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:25.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T17:27:25.034 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T17:27:25.035 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:25.036 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:25.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T17:27:25.347 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:25.412 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-rook 2024-09-17T17:27:25.422 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:25.486 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:25.487 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:27:25.488 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:25.488 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 154 k 2024-09-17T17:27:25.488 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:27:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T17:27:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:25.489 INFO:teuthology.orchestra.run.smithi026.stdout:Install 6 Packages 2024-09-17T17:27:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:25.491 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 7.0 M 2024-09-17T17:27:25.491 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 33 M 2024-09-17T17:27:25.491 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:25.561 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:27:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 899 k 2024-09-17T17:27:25.562 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 979 k 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 2.9 M 2024-09-17T17:27:25.563 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:25.764 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:25.821 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 623 kB/s | 80 kB 00:00 2024-09-17T17:27:25.904 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:25.904 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:27:25.904 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:25.904 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 154 k 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2024-09-17T17:27:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:25.906 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 7.0 M 2024-09-17T17:27:25.906 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 33 M 2024-09-17T17:27:25.906 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:25.979 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): ceph-fuse-19.3.0-4975.g1889a165.el9.x86_ 3.1 MB/s | 899 kB 00:00 2024-09-17T17:27:25.979 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:25.980 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.3 MB/s | 979 kB 00:00 2024-09-17T17:27:25.992 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:26.001 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:21 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:26.017 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:26.017 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:26.083 INFO:teuthology.orchestra.run.smithi026.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-17T17:27:26.102 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:26.103 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:26.108 INFO:teuthology.orchestra.run.smithi026.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4975.g1889a165.e 721 kB/s | 154 kB 00:00 2024-09-17T17:27:26.134 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 49 k 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T17:27:26.135 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:26.136 INFO:teuthology.orchestra.run.smithi102.stdout:Install 13 Packages 2024-09-17T17:27:26.137 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:26.137 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.2 M 2024-09-17T17:27:26.137 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 26 M 2024-09-17T17:27:26.137 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:26.176 INFO:teuthology.orchestra.run.smithi026.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 864 kB/s | 58 kB 00:00 2024-09-17T17:27:26.209 INFO:teuthology.orchestra.run.smithi026.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.3 MB/s | 548 kB 00:00 2024-09-17T17:27:26.260 INFO:teuthology.orchestra.run.smithi026.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 893 kB/s | 74 kB 00:00 2024-09-17T17:27:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:26.315 INFO:teuthology.orchestra.run.smithi102.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 334 kB/s | 26 kB 00:00 2024-09-17T17:27:26.332 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-17T17:27:26.340 INFO:teuthology.orchestra.run.smithi102.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 188 kB/s | 19 kB 00:00 2024-09-17T17:27:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-17T17:27:26.345 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:26.346 INFO:teuthology.orchestra.run.smithi026.stdout:Total 8.2 MB/s | 7.0 MB 00:00 2024-09-17T17:27:26.358 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4975.g1889a165.e 619 kB/s | 154 kB 00:00 2024-09-17T17:27:26.416 INFO:teuthology.orchestra.run.smithi102.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-17T17:27:26.416 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 991 kB/s | 58 kB 00:00 2024-09-17T17:27:26.419 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:26.427 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T17:27:26.431 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:26.431 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:26.441 INFO:teuthology.orchestra.run.smithi102.stdout:(4/13): ceph-mgr-rook-19.3.0-4975.g1889a165.el9 243 kB/s | 49 kB 00:00 2024-09-17T17:27:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.6 MB/s | 548 kB 00:00 2024-09-17T17:27:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-17T17:27:26.475 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-17T17:27:26.498 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:26.508 INFO:teuthology.orchestra.run.smithi102.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.0 MB/s | 279 kB 00:00 2024-09-17T17:27:26.533 INFO:teuthology.orchestra.run.smithi102.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-17T17:27:26.537 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:26.538 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:26.567 INFO:teuthology.orchestra.run.smithi102.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 426 kB/s | 54 kB 00:00 2024-09-17T17:27:26.601 INFO:teuthology.orchestra.run.smithi102.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-17T17:27:26.628 INFO:teuthology.orchestra.run.smithi102.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.5 MB/s | 220 kB 00:00 2024-09-17T17:27:26.651 INFO:teuthology.orchestra.run.smithi102.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-17T17:27:26.668 INFO:teuthology.orchestra.run.smithi102.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-17T17:27:26.710 INFO:teuthology.orchestra.run.smithi102.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-17T17:27:26.712 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:26.713 INFO:teuthology.orchestra.run.smithi102.stdout:Total 3.8 MB/s | 2.2 MB 00:00 2024-09-17T17:27:26.775 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:26.791 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:26.791 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:26.880 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:26.881 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T17:27:27.080 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T17:27:27.201 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:27.201 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/2 2024-09-17T17:27:27.384 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:27.513 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T17:27:27.513 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:27.513 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T17:27:27.514 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:27.514 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:27.556 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T17:27:27.602 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T17:27:27.653 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T17:27:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T17:27:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T17:27:27.728 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T17:27:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T17:27:27.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-volume 2024-09-17T17:27:27.827 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T17:27:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T17:27:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:27.931 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T17:27:27.988 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T17:27:28.422 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:28.560 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:28.561 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T17:27:28.561 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:28.561 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 285 k 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 285 k 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 1.3 M 2024-09-17T17:27:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:28.567 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-09-17T17:27:28.570 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:28.570 INFO:teuthology.orchestra.run.smithi027.stdout:Total 2.6 MB/s | 7.0 MB 00:02 2024-09-17T17:27:28.646 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:28.659 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:28.659 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:28.660 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:28.661 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/6 2024-09-17T17:27:28.661 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T17:27:28.661 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T17:27:28.661 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T17:27:28.661 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T17:27:28.691 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T17:27:28.752 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T17:27:28.768 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:28.772 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:28.786 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T17:27:28.835 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-volume-19.3.0-4975.g1889a165.el9.noarch.rp 1.0 MB/s | 285 kB 00:00 2024-09-17T17:27:28.835 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:28.836 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.0 MB/s | 285 kB 00:00 2024-09-17T17:27:28.836 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:28.845 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:28.845 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:28.850 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T17:27:28.871 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:28.872 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:28.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T17:27:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T17:27:28.944 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T17:27:28.944 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:28.944 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:29.090 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:29.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-fuse 2024-09-17T17:27:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:29.195 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:29.195 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T17:27:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:29.266 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T17:27:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T17:27:29.691 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 1/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T17:27:29.692 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T17:27:29.693 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T17:27:29.801 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:29.901 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T17:27:29.939 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 899 k 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T17:27:29.940 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:29.941 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:29.941 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:29.941 INFO:teuthology.orchestra.run.smithi026.stdout:Install 2 Packages 2024-09-17T17:27:29.941 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:29.941 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 979 k 2024-09-17T17:27:29.942 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 2.9 M 2024-09-17T17:27:29.942 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:29.983 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T17:27:30.025 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T17:27:30.025 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T17:27:30.026 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T17:27:30.027 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T17:27:30.027 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T17:27:30.027 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:30.027 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:30.059 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T17:27:30.089 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:30.228 INFO:teuthology.orchestra.run.smithi026.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-17T17:27:30.267 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T17:27:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:30.291 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:30.291 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:30.428 INFO:teuthology.orchestra.run.smithi026.stdout:(2/2): ceph-fuse-19.3.0-4975.g1889a165.el9.x86_ 2.6 MB/s | 899 kB 00:00 2024-09-17T17:27:30.429 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:30.429 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.0 MB/s | 979 kB 00:00 2024-09-17T17:27:30.442 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:30.466 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:30.466 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:30.555 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:30.556 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:30.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados-devel 2024-09-17T17:27:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:30.808 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:30.808 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/6 2024-09-17T17:27:30.808 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T17:27:30.808 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T17:27:30.808 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T17:27:30.808 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T17:27:30.845 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:26 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:30.893 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T17:27:30.973 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:30.977 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 154 k 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T17:27:30.978 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout:Install 6 Packages 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 7.0 M 2024-09-17T17:27:30.979 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 33 M 2024-09-17T17:27:30.980 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:31.097 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:31.214 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:31.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-fuse 2024-09-17T17:27:31.337 INFO:teuthology.orchestra.run.smithi102.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4975.g1889a165.e 569 kB/s | 154 kB 00:00 2024-09-17T17:27:31.352 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 126 k 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:31.353 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:31.354 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:27:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:31.354 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 126 k 2024-09-17T17:27:31.354 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 456 k 2024-09-17T17:27:31.354 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:31.363 INFO:teuthology.orchestra.run.smithi102.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 837 kB/s | 248 kB 00:00 2024-09-17T17:27:31.429 INFO:teuthology.orchestra.run.smithi102.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 866 kB/s | 58 kB 00:00 2024-09-17T17:27:31.471 INFO:teuthology.orchestra.run.smithi102.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-17T17:27:31.496 INFO:teuthology.orchestra.run.smithi102.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-17T17:27:31.574 INFO:teuthology.orchestra.run.smithi007.stdout:librados-devel-19.3.0-4975.g1889a165.el9.x86_64 576 kB/s | 126 kB 00:00 2024-09-17T17:27:31.575 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:31.575 INFO:teuthology.orchestra.run.smithi007.stdout:Total 572 kB/s | 126 kB 00:00 2024-09-17T17:27:31.575 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:31.581 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:31.581 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:31.640 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:31.640 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/2 2024-09-17T17:27:31.649 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:31.649 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:31.901 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T17:27:31.901 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:31.901 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:31.901 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T17:27:31.902 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:31.902 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:31.925 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:31.982 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:32.070 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 899 k 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout:Install 2 Packages 2024-09-17T17:27:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:32.072 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 979 k 2024-09-17T17:27:32.072 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 2.9 M 2024-09-17T17:27:32.072 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:32.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-volume 2024-09-17T17:27:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:32.427 INFO:teuthology.orchestra.run.smithi027.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 829 kB/s | 80 kB 00:00 2024-09-17T17:27:32.611 INFO:teuthology.orchestra.run.smithi027.stdout:(2/2): ceph-fuse-19.3.0-4975.g1889a165.el9.x86_ 3.1 MB/s | 899 kB 00:00 2024-09-17T17:27:32.611 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:32.612 INFO:teuthology.orchestra.run.smithi027.stdout:Total 1.8 MB/s | 979 kB 00:00 2024-09-17T17:27:32.624 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:32.647 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:32.647 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:32.667 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:32.667 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:32.735 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:32.735 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:32.803 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:32.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs2 2024-09-17T17:27:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:32.942 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 285 k 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:32.943 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:32.944 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:27:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:32.944 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 285 k 2024-09-17T17:27:32.944 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 1.3 M 2024-09-17T17:27:32.944 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T17:27:33.146 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:33.165 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-volume-19.3.0-4975.g1889a165.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-17T17:27:33.165 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:33.166 INFO:teuthology.orchestra.run.smithi026.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-17T17:27:33.166 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:33.175 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:33.175 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:33.203 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:33.204 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:33.517 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:33.518 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T17:27:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:33.519 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:33.546 INFO:teuthology.orchestra.run.smithi102.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-09-17T17:27:33.549 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:33.549 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.7 MB/s | 7.0 MB 00:02 2024-09-17T17:27:33.614 INFO:teuthology.orchestra.run.smithi007.stdout:Package libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:33.623 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:33.635 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:33.635 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:33.661 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:33.662 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:33.662 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:33.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs-devel 2024-09-17T17:27:33.736 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:33.737 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/2 2024-09-17T17:27:34.046 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:34.117 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T17:27:34.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:34.117 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:34.117 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T17:27:34.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:34.117 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:34.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T17:27:34.264 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T17:27:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:34.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-volume 2024-09-17T17:27:34.484 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:34.484 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:34.484 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:34.484 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:34.485 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:34.485 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:34.486 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:34.487 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T17:27:34.487 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:34.487 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 32 k 2024-09-17T17:27:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:34.488 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:34.488 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:34.488 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:27:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:34.488 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 32 k 2024-09-17T17:27:34.489 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 142 k 2024-09-17T17:27:34.489 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:34.647 INFO:teuthology.orchestra.run.smithi007.stdout:libcephfs-devel-19.3.0-4975.g1889a165.el9.x86_6 201 kB/s | 32 kB 00:00 2024-09-17T17:27:34.647 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:34.648 INFO:teuthology.orchestra.run.smithi007.stdout:Total 198 kB/s | 32 kB 00:00 2024-09-17T17:27:34.648 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:34.652 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:34.652 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:34.666 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:34.666 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:34.780 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T17:27:34.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install librados-devel 2024-09-17T17:27:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:34.867 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T17:27:34.943 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T17:27:34.969 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:35.042 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:35.180 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 285 k 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:35.181 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:35.182 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:35.182 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:27:35.182 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:35.182 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 285 k 2024-09-17T17:27:35.182 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 1.3 M 2024-09-17T17:27:35.182 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:35.390 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-volume-19.3.0-4975.g1889a165.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-17T17:27:35.391 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:35.391 INFO:teuthology.orchestra.run.smithi027.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-17T17:27:35.391 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:35.392 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:35.400 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:35.400 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:35.427 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:35.428 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:35.531 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 126 k 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 126 k 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 456 k 2024-09-17T17:27:35.532 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:35.541 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:35.542 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:35.662 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:35.719 INFO:teuthology.orchestra.run.smithi026.stdout:librados-devel-19.3.0-4975.g1889a165.el9.x86_64 678 kB/s | 126 kB 00:00 2024-09-17T17:27:35.720 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:35.720 INFO:teuthology.orchestra.run.smithi026.stdout:Total 673 kB/s | 126 kB 00:00 2024-09-17T17:27:35.720 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:35.726 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:35.726 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:35.734 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:35.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados2 2024-09-17T17:27:35.751 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:35.751 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:35.751 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T17:27:35.751 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:35.795 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:35.795 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:35.837 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T17:27:35.837 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/6 2024-09-17T17:27:35.837 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T17:27:35.837 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T17:27:35.837 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T17:27:35.837 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T17:27:35.967 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:36.078 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:36.117 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:36.353 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:36.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-fuse 2024-09-17T17:27:36.445 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:36.492 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:36.493 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:36.493 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:36.503 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:36.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librbd1 2024-09-17T17:27:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:36.748 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:36.748 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:36.755 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:36.755 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:36.755 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:36.755 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:36.755 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:36.755 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:36.938 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:36.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install libcephfs2 2024-09-17T17:27:37.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados-devel 2024-09-17T17:27:37.070 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:37.071 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:37.071 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:27:37.071 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 899 k 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:Install 2 Packages 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 979 k 2024-09-17T17:27:37.072 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.9 M 2024-09-17T17:27:37.073 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:37.159 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:37.253 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:37.299 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:37.300 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:37.300 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:37.356 INFO:teuthology.orchestra.run.smithi102.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 515 kB/s | 80 kB 00:00 2024-09-17T17:27:37.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rados 2024-09-17T17:27:37.508 INFO:teuthology.orchestra.run.smithi102.stdout:(2/2): ceph-fuse-19.3.0-4975.g1889a165.el9.x86_ 2.9 MB/s | 899 kB 00:00 2024-09-17T17:27:37.509 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:37.509 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.2 MB/s | 979 kB 00:00 2024-09-17T17:27:37.521 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:37.561 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:37.623 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:37.623 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:37.642 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:37.654 INFO:teuthology.orchestra.run.smithi026.stdout:Package libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:37.700 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:37.701 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:37.701 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:37.779 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:37.780 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:37.780 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-17T17:27:37.780 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:37.780 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 126 k 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 126 k 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 456 k 2024-09-17T17:27:37.781 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:37.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install libcephfs-devel 2024-09-17T17:27:37.804 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:37.944 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T17:27:37.969 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:38.005 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:38.063 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:38.112 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:38.113 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:38.113 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:38.153 INFO:teuthology.orchestra.run.smithi027.stdout:librados-devel-19.3.0-4975.g1889a165.el9.x86_64 340 kB/s | 126 kB 00:00 2024-09-17T17:27:38.153 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:38.154 INFO:teuthology.orchestra.run.smithi027.stdout:Total 339 kB/s | 126 kB 00:00 2024-09-17T17:27:38.154 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:38.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rgw 2024-09-17T17:27:38.229 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:38.230 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:38.388 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:38.401 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:38.504 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:38.527 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:38.528 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:38.528 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-09-17T17:27:38.528 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:38.528 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 32 k 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 32 k 2024-09-17T17:27:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 142 k 2024-09-17T17:27:38.530 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:38.706 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T17:27:38.706 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/2 2024-09-17T17:27:38.777 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:38.814 INFO:teuthology.orchestra.run.smithi026.stdout:libcephfs-devel-19.3.0-4975.g1889a165.el9.x86_6 111 kB/s | 32 kB 00:00 2024-09-17T17:27:38.815 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:38.815 INFO:teuthology.orchestra.run.smithi026.stdout:Total 111 kB/s | 32 kB 00:00 2024-09-17T17:27:38.815 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:38.819 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:38.819 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:38.834 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:38.834 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:38.869 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:38.916 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:38.916 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:38.917 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:38.917 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:38.965 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:38.975 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T17:27:38.976 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:38.976 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:38.976 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T17:27:38.976 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:38.976 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:38.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-cephfs 2024-09-17T17:27:39.031 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:39.227 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-volume 2024-09-17T17:27:39.229 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:39.229 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:39.229 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:39.230 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:39.380 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:39.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs2 2024-09-17T17:27:39.586 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:39.615 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:39.615 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:39.615 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:39.615 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:39.615 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:39.615 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:39.679 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:39.726 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:39.727 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:39.727 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:39.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rbd 2024-09-17T17:27:39.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install librados2 2024-09-17T17:27:39.814 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:39.956 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 285 k 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:39.957 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 285 k 2024-09-17T17:27:39.958 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.3 M 2024-09-17T17:27:39.958 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:40.187 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:40.281 INFO:teuthology.orchestra.run.smithi027.stdout:Package libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:40.326 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:40.327 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:40.328 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:40.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-volume-19.3.0-4975.g1889a165.el9.noarch.rp 678 kB/s | 285 kB 00:00 2024-09-17T17:27:40.379 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:40.379 INFO:teuthology.orchestra.run.smithi102.stdout:Total 676 kB/s | 285 kB 00:00 2024-09-17T17:27:40.379 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:40.387 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:40.388 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:40.391 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:40.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs-devel 2024-09-17T17:27:40.403 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:40.419 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:40.420 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:40.484 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:40.495 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:40.530 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:40.531 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T17:27:40.531 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:40.543 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:40.543 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:40.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-fuse 2024-09-17T17:27:40.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install librbd1 2024-09-17T17:27:40.641 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:40.713 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:40.729 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:40.729 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:40.729 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T17:27:40.729 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:40.999 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:41.140 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:41.140 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:41.140 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 32 k 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:27:41.141 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:41.142 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 32 k 2024-09-17T17:27:41.142 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 142 k 2024-09-17T17:27:41.142 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:41.229 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:41.321 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:41.335 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:41.335 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 85 k 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:27:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:41.337 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 85 k 2024-09-17T17:27:41.337 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 231 k 2024-09-17T17:27:41.337 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:41.367 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:41.369 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:41.369 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:41.416 INFO:teuthology.orchestra.run.smithi027.stdout:libcephfs-devel-19.3.0-4975.g1889a165.el9.x86_6 116 kB/s | 32 kB 00:00 2024-09-17T17:27:41.416 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:41.416 INFO:teuthology.orchestra.run.smithi027.stdout:Total 115 kB/s | 32 kB 00:00 2024-09-17T17:27:41.416 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:41.420 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:41.420 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:41.441 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:41.441 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:41.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-rados 2024-09-17T17:27:41.523 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:41.633 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:41.717 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-fuse-19.3.0-4975.g1889a165.el9.x86_64.rpm 224 kB/s | 85 kB 00:00 2024-09-17T17:27:41.718 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:41.718 INFO:teuthology.orchestra.run.smithi007.stdout:Total 223 kB/s | 85 kB 00:00 2024-09-17T17:27:41.718 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:41.725 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:41.726 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:41.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T17:27:41.757 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:41.757 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:41.757 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T17:27:41.757 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:41.757 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:41.795 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:42.021 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:42.050 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:42.058 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados-devel 2024-09-17T17:27:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:42.142 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:42.188 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:42.189 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:42.189 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:42.248 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:42.248 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:42.248 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:42.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-rgw 2024-09-17T17:27:42.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados2 2024-09-17T17:27:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:42.655 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:42.746 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:42.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:42.746 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:42.746 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:42.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:42.747 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:42.787 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 126 k 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:42.788 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:42.789 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:42.789 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:27:42.789 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:42.789 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 126 k 2024-09-17T17:27:42.789 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 456 k 2024-09-17T17:27:42.789 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:42.867 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:42.960 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:43.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-mirror 2024-09-17T17:27:43.006 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:43.007 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:43.007 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:43.011 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:43.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-cephfs 2024-09-17T17:27:43.103 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:43.149 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:43.150 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:43.150 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:43.165 INFO:teuthology.orchestra.run.smithi102.stdout:librados-devel-19.3.0-4975.g1889a165.el9.x86_64 336 kB/s | 126 kB 00:00 2024-09-17T17:27:43.166 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:43.167 INFO:teuthology.orchestra.run.smithi102.stdout:Total 335 kB/s | 126 kB 00:00 2024-09-17T17:27:43.167 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:43.172 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:43.173 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:43.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librbd1 2024-09-17T17:27:43.237 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:43.237 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:43.399 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:43.506 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:43.621 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:43.697 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:43.763 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.2 M 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:27:43.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 3.2 M 2024-09-17T17:27:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 14 M 2024-09-17T17:27:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:43.791 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:43.826 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:43.837 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:43.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-rbd 2024-09-17T17:27:43.919 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:43.965 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:43.966 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:43.966 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:44.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rados 2024-09-17T17:27:44.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:44.275 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:44.275 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:44.275 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:44.275 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:44.276 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:44.518 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:44.527 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs2 2024-09-17T17:27:44.597 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-mirror-19.3.0-4975.g1889a165.el9.x86_64.rpm 3.8 MB/s | 3.2 MB 00:00 2024-09-17T17:27:44.597 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:44.598 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.8 MB/s | 3.2 MB 00:00 2024-09-17T17:27:44.598 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:44.609 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:44.609 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:44.610 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:44.635 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:44.657 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:44.658 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-09-17T17:27:44.658 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:44.697 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:44.728 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:44.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install rbd-fuse 2024-09-17T17:27:44.774 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:44.775 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:44.775 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:44.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rgw 2024-09-17T17:27:45.135 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:45.230 INFO:teuthology.orchestra.run.smithi102.stdout:Package libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:45.231 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:45.258 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:45.258 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:45.259 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T17:27:45.259 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:45.259 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:45.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:45.274 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:45.275 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:45.275 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:45.336 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs-devel 2024-09-17T17:27:45.358 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:45.447 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:45.497 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:45.498 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:45.498 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:45.498 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:45.498 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 85 k 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 85 k 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 231 k 2024-09-17T17:27:45.499 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:45.539 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:45.585 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:45.586 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:45.587 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:45.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-cephfs 2024-09-17T17:27:45.867 INFO:teuthology.orchestra.run.smithi026.stdout:rbd-fuse-19.3.0-4975.g1889a165.el9.x86_64.rpm 231 kB/s | 85 kB 00:00 2024-09-17T17:27:45.867 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:45.868 INFO:teuthology.orchestra.run.smithi026.stdout:Total 231 kB/s | 85 kB 00:00 2024-09-17T17:27:45.868 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:45.875 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:45.876 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:45.911 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:45.943 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:45.944 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:46.043 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 32 k 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:46.044 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:46.045 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:46.045 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:27:46.045 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:46.045 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 32 k 2024-09-17T17:27:46.045 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 142 k 2024-09-17T17:27:46.045 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:46.098 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:46.193 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:46.258 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:46.329 INFO:teuthology.orchestra.run.smithi102.stdout:libcephfs-devel-19.3.0-4975.g1889a165.el9.x86_6 112 kB/s | 32 kB 00:00 2024-09-17T17:27:46.329 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:46.330 INFO:teuthology.orchestra.run.smithi102.stdout:Total 111 kB/s | 32 kB 00:00 2024-09-17T17:27:46.330 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:46.333 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:46.333 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:46.346 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:46.346 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:46.350 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:46.396 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:46.397 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:46.397 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:46.421 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:46.421 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:46.437 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:46.471 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rbd 2024-09-17T17:27:46.543 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:46.584 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:46.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-09-17T17:27:46.866 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:46.867 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:46.867 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:46.867 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:46.915 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:47.075 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:47.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install rbd-mirror 2024-09-17T17:27:47.155 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:47.155 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:47.155 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:47.155 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:47.155 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:47.155 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:47.168 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:47.215 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:47.216 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-17T17:27:47.216 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:47.285 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-fuse 2024-09-17T17:27:47.285 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:58 ago on Tue 17 Sep 2024 05:25:49 PM UTC. 2024-09-17T17:27:47.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2 2024-09-17T17:27:47.430 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 170 k 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T17:27:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:47.432 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 170 k 2024-09-17T17:27:47.432 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 490 k 2024-09-17T17:27:47.432 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T17:27:47.738 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:56 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:47.818 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-nbd-19.3.0-4975.g1889a165.el9.x86_64.rpm 441 kB/s | 170 kB 00:00 2024-09-17T17:27:47.818 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:47.819 INFO:teuthology.orchestra.run.smithi007.stdout:Total 440 kB/s | 170 kB 00:00 2024-09-17T17:27:47.819 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T17:27:47.827 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T17:27:47.827 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T17:27:47.878 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:47.878 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:47.879 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-09-17T17:27:47.879 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:47.879 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:47.879 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.2 M 2024-09-17T17:27:47.879 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:47.879 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:47.880 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:47.880 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:27:47.880 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:47.880 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 3.2 M 2024-09-17T17:27:47.880 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 14 M 2024-09-17T17:27:47.880 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:47.889 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:47.896 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T17:27:47.896 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T17:27:47.930 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:48.019 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:48.028 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:48.028 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:48.028 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:48.028 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:48.028 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:48.028 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 85 k 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 85 k 2024-09-17T17:27:48.029 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 231 k 2024-09-17T17:27:48.030 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:48.062 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:48.063 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:48.063 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:48.125 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librbd1 2024-09-17T17:27:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T17:27:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:48.369 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-fuse-19.3.0-4975.g1889a165.el9.x86_64.rpm 251 kB/s | 85 kB 00:00 2024-09-17T17:27:48.369 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:48.369 INFO:teuthology.orchestra.run.smithi027.stdout:Total 250 kB/s | 85 kB 00:00 2024-09-17T17:27:48.370 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:48.377 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:48.377 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:48.444 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:48.445 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:48.604 INFO:teuthology.orchestra.run.smithi026.stdout:rbd-mirror-19.3.0-4975.g1889a165.el9.x86_64.rpm 4.4 MB/s | 3.2 MB 00:00 2024-09-17T17:27:48.605 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:48.605 INFO:teuthology.orchestra.run.smithi026.stdout:Total 4.4 MB/s | 3.2 MB 00:00 2024-09-17T17:27:48.605 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:48.616 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:48.616 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:48.702 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:48.702 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:48.703 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:48.713 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:48.792 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:48.828 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:48.834 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:48.835 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:48.835 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:48.902 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rados 2024-09-17T17:27:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:49.046 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T17:27:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:27:49.046 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T17:27:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:49.221 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:49.249 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:49.249 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:49.249 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T17:27:49.249 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:49.249 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:49.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:49.287 DEBUG:teuthology.parallel:result is None 2024-09-17T17:27:49.347 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:49.482 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:49.571 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:49.593 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:49.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:49.593 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:49.615 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:49.616 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:49.616 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:49.676 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rgw 2024-09-17T17:27:49.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-mirror 2024-09-17T17:27:50.259 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:50.348 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:50.390 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:50.391 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:50.391 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:50.391 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:50.391 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:50.393 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:50.393 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:50.445 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:57 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:50.460 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-cephfs 2024-09-17T17:27:50.585 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.2 M 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:50.586 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:50.587 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:50.587 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:27:50.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:50.587 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.2 M 2024-09-17T17:27:50.587 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 14 M 2024-09-17T17:27:50.587 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:50.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install rbd-nbd 2024-09-17T17:27:50.972 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-mirror-19.3.0-4975.g1889a165.el9.x86_64.rpm 8.3 MB/s | 3.2 MB 00:00 2024-09-17T17:27:50.974 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:50.974 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.2 MB/s | 3.2 MB 00:00 2024-09-17T17:27:50.974 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:50.984 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:50.984 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:51.036 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:51.070 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:51.070 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:51.126 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:51.170 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:51.171 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:51.171 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:51.233 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rbd 2024-09-17T17:27:51.239 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:00 ago on Tue 17 Sep 2024 05:25:51 PM UTC. 2024-09-17T17:27:51.377 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-17T17:27:51.378 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:51.378 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 170 k 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 170 k 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 490 k 2024-09-17T17:27:51.379 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-17T17:27:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:51.582 INFO:teuthology.orchestra.run.smithi026.stdout:rbd-nbd-19.3.0-4975.g1889a165.el9.x86_64.rpm 838 kB/s | 170 kB 00:00 2024-09-17T17:27:51.583 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:51.583 INFO:teuthology.orchestra.run.smithi026.stdout:Total 832 kB/s | 170 kB 00:00 2024-09-17T17:27:51.583 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-17T17:27:51.592 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-17T17:27:51.592 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-17T17:27:51.606 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:51.633 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:51.633 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:51.633 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T17:27:51.633 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:51.634 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:51.634 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:51.661 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-17T17:27:51.661 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-17T17:27:51.812 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-17T17:27:51.901 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T17:27:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:51.945 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:51.946 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-17T17:27:51.946 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:52.008 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-fuse 2024-09-17T17:27:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:52.586 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:52.608 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-17T17:27:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:27:52.608 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-17T17:27:52.719 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:52.719 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 85 k 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 85 k 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 231 k 2024-09-17T17:27:52.720 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:52.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:52.742 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:52.742 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:52.743 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:52.743 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:52.743 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:52.836 DEBUG:teuthology.parallel:result is None 2024-09-17T17:27:52.923 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-fuse-19.3.0-4975.g1889a165.el9.x86_64.rpm 420 kB/s | 85 kB 00:00 2024-09-17T17:27:52.924 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:52.924 INFO:teuthology.orchestra.run.smithi102.stdout:Total 417 kB/s | 85 kB 00:00 2024-09-17T17:27:52.924 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:52.931 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:52.931 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:52.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-nbd 2024-09-17T17:27:52.994 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:52.994 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:53.140 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:53.246 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:53.576 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:00 ago on Tue 17 Sep 2024 05:25:53 PM UTC. 2024-09-17T17:27:53.606 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:53.715 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-17T17:27:53.716 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:53.716 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:53.716 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:53.716 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-17T17:27:53.716 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 170 k 2024-09-17T17:27:53.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 170 k 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 490 k 2024-09-17T17:27:53.717 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-17T17:27:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:53.855 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:53.856 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:53.856 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:53.925 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-nbd-19.3.0-4975.g1889a165.el9.x86_64.rpm 818 kB/s | 170 kB 00:00 2024-09-17T17:27:53.926 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:53.926 INFO:teuthology.orchestra.run.smithi027.stdout:Total 811 kB/s | 170 kB 00:00 2024-09-17T17:27:53.926 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-17T17:27:53.935 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-17T17:27:53.935 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-17T17:27:54.003 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-17T17:27:54.003 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-17T17:27:54.127 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-mirror 2024-09-17T17:27:54.163 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-17T17:27:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:54.678 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:54.722 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.2 M 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:54.858 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:54.859 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:54.859 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:27:54.859 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:54.859 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 3.2 M 2024-09-17T17:27:54.860 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 14 M 2024-09-17T17:27:54.860 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:54.926 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:54.926 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:54.926 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-17T17:27:54.926 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:54.926 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:27:54.926 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-17T17:27:55.133 DEBUG:teuthology.parallel:result is None 2024-09-17T17:27:55.193 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-mirror-19.3.0-4975.g1889a165.el9.x86_64.rpm 9.5 MB/s | 3.2 MB 00:00 2024-09-17T17:27:55.194 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:55.194 INFO:teuthology.orchestra.run.smithi102.stdout:Total 9.5 MB/s | 3.2 MB 00:00 2024-09-17T17:27:55.195 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:55.203 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:55.204 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:55.285 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:55.285 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:55.737 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:55.786 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:55.812 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:55.812 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T17:27:55.812 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T17:27:55.812 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:55.812 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T17:27:55.812 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:56.991 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:56.991 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:56.991 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:56.991 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:56.991 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:56.992 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:57.233 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-nbd 2024-09-17T17:27:57.825 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 05:26:04 PM UTC. 2024-09-17T17:27:57.959 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 170 k 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:57.960 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 170 k 2024-09-17T17:27:57.961 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 490 k 2024-09-17T17:27:57.961 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-17T17:27:58.172 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-nbd-19.3.0-4975.g1889a165.el9.x86_64.rpm 805 kB/s | 170 kB 00:00 2024-09-17T17:27:58.173 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-17T17:27:58.173 INFO:teuthology.orchestra.run.smithi102.stdout:Total 800 kB/s | 170 kB 00:00 2024-09-17T17:27:58.173 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-17T17:27:58.180 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-17T17:27:58.181 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-17T17:27:58.244 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-17T17:27:58.244 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-17T17:27:58.407 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-17T17:27:58.512 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:58.936 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:59.197 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T17:27:59.197 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-17T17:27:59.197 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T17:27:59.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:27:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-17T17:27:59.459 DEBUG:teuthology.parallel:result is None 2024-09-17T17:27:59.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:27:59.613 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T17:27:59.640 INFO:teuthology.orchestra.run.smithi007.stdout:19.3.0-4975.g1889a165.el9 2024-09-17T17:27:59.641 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g1889a165.el9 2024-09-17T17:27:59.641 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g1889a165 is installed. 2024-09-17T17:27:59.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:27:59.789 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T17:27:59.815 INFO:teuthology.orchestra.run.smithi026.stdout:19.3.0-4975.g1889a165.el9 2024-09-17T17:27:59.815 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g1889a165.el9 2024-09-17T17:27:59.816 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g1889a165 is installed. 2024-09-17T17:27:59.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:27:59.966 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T17:27:59.993 INFO:teuthology.orchestra.run.smithi027.stdout:19.3.0-4975.g1889a165.el9 2024-09-17T17:27:59.993 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g1889a165.el9 2024-09-17T17:27:59.994 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g1889a165 is installed. 2024-09-17T17:27:59.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T17:28:00.163 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T17:28:00.186 INFO:teuthology.orchestra.run.smithi102.stdout:19.3.0-4975.g1889a165.el9 2024-09-17T17:28:00.187 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g1889a165.el9 2024-09-17T17:28:00.187 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g1889a165 is installed. 2024-09-17T17:28:00.188 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T17:28:00.189 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:00.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T17:28:00.221 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:00.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T17:28:00.252 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:00.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T17:28:00.285 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:00.285 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T17:28:00.312 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T17:28:00.312 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:00.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T17:28:00.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T17:28:00.413 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:00.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T17:28:00.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T17:28:00.515 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:00.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T17:28:00.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T17:28:00.616 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:00.616 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T17:28:00.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T17:28:00.711 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T17:28:00.712 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:00.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T17:28:00.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T17:28:00.813 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:00.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T17:28:00.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T17:28:00.913 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:00.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T17:28:00.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T17:28:01.014 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:01.014 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T17:28:01.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T17:28:01.110 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T17:28:01.111 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:01.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T17:28:01.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T17:28:01.212 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:01.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T17:28:01.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T17:28:01.313 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:01.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T17:28:01.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T17:28:01.415 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:01.415 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T17:28:01.444 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T17:28:01.512 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T17:28:01.591 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T17:28:01.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-17T17:28:01.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-09-17T17:28:01.597 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2024-09-17T17:28:01.600 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2024-09-17T17:28:01.628 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T17:28:01.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T17:28:01.665 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T17:28:01.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T17:28:01.671 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T17:28:01.698 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T17:28:01.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T17:28:01.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T17:28:01.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T17:28:01.741 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T17:28:01.779 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T17:28:01.779 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T17:28:01.779 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156', 'branch': 'wip-bharath10-testing-2024-09-17-1348', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-hybrid} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7909156', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath10-testing-2024-09-17-1348'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath10-testing-2024-09-17-1348', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 3421, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath10-testing-2024-09-17-1348', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXIDGWWnEt9ojuP6e23K4bQQwPKkBm2z5+4ShXYpsiSSjlgI5YVqM2pBwJ26fcJftxvQp8hL0gAiNDfjpexmJ8=', 'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFeZUbjSYefjq3WII5LE+SUxbHhqvlMBj2hyAD3HjQCVfS3IsozictrBx0nZ/UVzvcA7VtGWHeHQ+d7vYtTLeE=', 'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBUyqqqCxlxFiiUg4s+MWqz5xKdSLMrLuiohJ+HD9cZcpSNmVPvn9aS5oPSfMXrlBDErbS037L4Tlnw2L5LOpQg=', 'smithi102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCp51lPnWmMwV1Js9tykoeu4DQ8nr+s+UJZqCq4ie4VrzHfSAFva3YFG0EsLBMq+y0gEIojWyVXuleVlWjUPiWo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'cluster': 'ceph'}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_15:01:33', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T17:28:01.780 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T17:28:01.806 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T17:28:01.808 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T17:28:01.811 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T17:28:01.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T17:28:01.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T17:28:01.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T17:28:01.869 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T17:28:01.895 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:01.895 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T17:28:01.948 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T17:28:01.948 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-17T17:28:02.005 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T17:28:02.005 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:02.005 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-17T17:28:02.005 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:02.005 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:02.006 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 17:27:45.505650818 +0000 2024-09-17T17:28:02.006 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 17:24:58.984351415 +0000 2024-09-17T17:28:02.006 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 17:24:58.984351415 +0000 2024-09-17T17:28:02.006 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 17:24:58.984351415 +0000 2024-09-17T17:28:02.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T17:28:02.071 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T17:28:02.071 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T17:28:02.071 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00010132 s, 5.1 MB/s 2024-09-17T17:28:02.072 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T17:28:02.130 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-17T17:28:02.186 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T17:28:02.186 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:02.186 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-17T17:28:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 17:27:45.506650802 +0000 2024-09-17T17:28:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 17:24:59.311346201 +0000 2024-09-17T17:28:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 17:24:59.311346201 +0000 2024-09-17T17:28:02.187 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 17:24:59.311346201 +0000 2024-09-17T17:28:02.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T17:28:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T17:28:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T17:28:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.9439e-05 s, 5.1 MB/s 2024-09-17T17:28:02.254 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T17:28:02.311 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-17T17:28:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T17:28:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 17:27:45.507650786 +0000 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 17:24:59.632341081 +0000 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 17:24:59.632341081 +0000 2024-09-17T17:28:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 17:24:59.632341081 +0000 2024-09-17T17:28:02.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T17:28:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T17:28:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T17:28:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 8.859e-05 s, 5.8 MB/s 2024-09-17T17:28:02.434 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T17:28:02.491 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-17T17:28:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T17:28:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:02.548 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-17T17:28:02.548 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:02.548 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:02.548 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 17:27:45.507650786 +0000 2024-09-17T17:28:02.549 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 17:24:59.956335915 +0000 2024-09-17T17:28:02.549 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 17:24:59.956335915 +0000 2024-09-17T17:28:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 17:24:59.956335915 +0000 2024-09-17T17:28:02.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T17:28:02.612 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T17:28:02.612 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T17:28:02.612 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.8279e-05 s, 5.2 MB/s 2024-09-17T17:28:02.613 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T17:28:02.671 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:02.671 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:02.671 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T17:28:02.691 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T17:28:02.691 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-09-17T17:28:02.750 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-17 17:27:49.520438516 +0000 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-17 17:24:57.499844107 +0000 2024-09-17T17:28:02.751 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-17 17:24:57.499844107 +0000 2024-09-17T17:28:02.752 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-17 17:24:57.499844107 +0000 2024-09-17T17:28:02.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T17:28:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-17T17:28:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-17T17:28:02.820 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000111783 s, 4.6 MB/s 2024-09-17T17:28:02.821 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T17:28:02.881 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-17 17:27:49.520438516 +0000 2024-09-17T17:28:02.940 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-17 17:24:57.837839192 +0000 2024-09-17T17:28:02.941 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-17 17:24:57.837839192 +0000 2024-09-17T17:28:02.941 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-17 17:24:57.837839192 +0000 2024-09-17T17:28:02.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T17:28:03.007 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-17T17:28:03.008 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-17T17:28:03.008 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000113287 s, 4.5 MB/s 2024-09-17T17:28:03.009 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T17:28:03.068 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-09-17T17:28:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T17:28:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-17 17:27:49.520438516 +0000 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-17 17:24:58.174834291 +0000 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-17 17:24:58.174834291 +0000 2024-09-17T17:28:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-17 17:24:58.174834291 +0000 2024-09-17T17:28:03.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T17:28:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-17T17:28:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-17T17:28:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000112368 s, 4.6 MB/s 2024-09-17T17:28:03.194 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T17:28:03.252 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-09-17T17:28:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T17:28:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-17 17:27:49.521438502 +0000 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-17 17:24:58.517829304 +0000 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-17 17:24:58.517829304 +0000 2024-09-17T17:28:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-17 17:24:58.517829304 +0000 2024-09-17T17:28:03.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T17:28:03.377 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-17T17:28:03.377 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-17T17:28:03.377 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000117365 s, 4.4 MB/s 2024-09-17T17:28:03.378 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T17:28:03.436 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:03.437 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:03.437 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T17:28:03.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T17:28:03.457 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-17T17:28:03.516 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-17 17:27:51.893205447 +0000 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-17 17:25:07.735680599 +0000 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-17 17:25:07.735680599 +0000 2024-09-17T17:28:03.517 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-17 17:25:07.735680599 +0000 2024-09-17T17:28:03.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T17:28:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-17T17:28:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-17T17:28:03.586 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000280678 s, 1.8 MB/s 2024-09-17T17:28:03.587 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T17:28:03.647 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-17T17:28:03.705 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-17 17:27:51.893205447 +0000 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-17 17:25:08.083671158 +0000 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-17 17:25:08.083671158 +0000 2024-09-17T17:28:03.706 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-17 17:25:08.083671158 +0000 2024-09-17T17:28:03.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T17:28:03.773 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-17T17:28:03.774 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-17T17:28:03.774 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000224288 s, 2.3 MB/s 2024-09-17T17:28:03.775 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T17:28:03.835 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-17T17:28:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T17:28:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-17 17:27:51.893205447 +0000 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-17 17:25:08.436661582 +0000 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-17 17:25:08.436661582 +0000 2024-09-17T17:28:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-17 17:25:08.436661582 +0000 2024-09-17T17:28:03.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T17:28:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-17T17:28:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-17T17:28:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000195425 s, 2.6 MB/s 2024-09-17T17:28:03.962 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T17:28:04.020 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-17T17:28:04.077 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T17:28:04.077 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:04.077 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-17T17:28:04.077 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:04.078 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:04.078 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-17 17:27:51.894205420 +0000 2024-09-17T17:28:04.078 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-17 17:25:08.787652060 +0000 2024-09-17T17:28:04.078 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-17 17:25:08.787652060 +0000 2024-09-17T17:28:04.078 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-17 17:25:08.787652060 +0000 2024-09-17T17:28:04.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T17:28:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-17T17:28:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-17T17:28:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000214057 s, 2.4 MB/s 2024-09-17T17:28:04.145 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T17:28:04.203 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:04.203 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:04.203 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T17:28:04.222 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T17:28:04.222 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-17T17:28:04.279 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-17 17:27:56.066200481 +0000 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-17 17:24:54.306899931 +0000 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-17 17:24:54.306899931 +0000 2024-09-17T17:28:04.280 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-17 17:24:54.306899931 +0000 2024-09-17T17:28:04.280 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T17:28:04.347 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-17T17:28:04.347 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-17T17:28:04.347 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000228656 s, 2.2 MB/s 2024-09-17T17:28:04.348 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T17:28:04.408 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-17 17:27:56.066200481 +0000 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-17 17:24:54.643892709 +0000 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-17 17:24:54.643892709 +0000 2024-09-17T17:28:04.467 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-17 17:24:54.643892709 +0000 2024-09-17T17:28:04.468 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T17:28:04.534 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-17T17:28:04.534 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-17T17:28:04.535 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000305388 s, 1.7 MB/s 2024-09-17T17:28:04.536 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T17:28:04.595 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-17T17:28:04.652 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T17:28:04.652 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:04.652 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T17:28:04.652 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:04.652 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:04.652 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-17 17:27:56.066200481 +0000 2024-09-17T17:28:04.653 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-17 17:24:54.984885402 +0000 2024-09-17T17:28:04.653 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-17 17:24:54.984885402 +0000 2024-09-17T17:28:04.653 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-17 17:24:54.984885402 +0000 2024-09-17T17:28:04.653 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T17:28:04.719 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-17T17:28:04.719 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-17T17:28:04.719 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000218993 s, 2.3 MB/s 2024-09-17T17:28:04.720 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T17:28:04.779 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-17T17:28:04.836 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T17:28:04.836 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T17:28:04.836 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-17T17:28:04.837 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T17:28:04.837 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T17:28:04.837 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-17 17:27:56.067200460 +0000 2024-09-17T17:28:04.837 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-17 17:24:55.309878437 +0000 2024-09-17T17:28:04.837 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-17 17:24:55.309878437 +0000 2024-09-17T17:28:04.837 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-17 17:24:55.309878437 +0000 2024-09-17T17:28:04.837 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T17:28:04.902 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-17T17:28:04.902 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-17T17:28:04.902 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000220792 s, 2.3 MB/s 2024-09-17T17:28:04.903 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T17:28:04.961 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:04.961 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-17T17:28:04.961 INFO:tasks.ceph:Generating config... 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] bluestore write v2 = False 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T17:28:04.963 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-17T17:28:04.964 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T17:28:04.965 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-17T17:28:04.965 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-17T17:28:04.965 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-17T17:28:04.965 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-17T17:28:04.965 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] ms inject delay max = 1 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] ms inject delay type = osd 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-09-17T17:28:04.965 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-09-17T17:28:04.966 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-09-17T17:28:04.966 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T17:28:04.966 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T17:28:04.966 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T17:28:04.966 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T17:28:04.966 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T17:28:04.966 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-17T17:28:04.966 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T17:28:04.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T17:28:05.042 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T17:28:05.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T17:28:05.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T17:28:05.121 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.26'), ('mon.c', '172.21.15.27')] 2024-09-17T17:28:05.121 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.26,172.21.15.27', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-17T17:28:05.122 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:05.122 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T17:28:05.177 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.26 --add c 172.21.15.27 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-17T17:28:05.265807+0000 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-17T17:28:05.265807+0000 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-17T17:28:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-17T17:28:05.268 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.c 2024-09-17T17:28:05.268 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T17:28:05.269 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T17:28:05.285 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2593b77f-a53d-4649-b5a5-0122bd31e2b3... 2024-09-17T17:28:05.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T17:28:05.327 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T17:28:05.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T17:28:05.332 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T17:28:05.377 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-17T17:28:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-17T17:28:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T17:28:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-17T17:28:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-17T17:28:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-17T17:28:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-17T17:28:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.26,172.21.15.27" 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: mon election default strategy = 1 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay max = 1 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay probability = 0.005 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay type = osd 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject internal delays = 0.002 2024-09-17T17:28:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 2500 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd_async_recovery_min_cost = 1 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 120 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-17T17:28:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-17T17:28:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: debug monc = 20 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: bluefs allocator = hybrid 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = hybrid 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-17T17:28:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore write v2 = False 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore zero block detection = True 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: osd blocked scrub grace period = 3600 2024-09-17T17:28:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug reject backfill probability = 0.3 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify cached snaps = True 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify missing on start = True 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd max backfills = 3 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd max markdown count = 1000 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock override recovery settings = True 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd op queue = debug_random 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd op queue cut off = debug_random 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub min interval = 60 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: osd shutdown pgref assert = True 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.386 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-17T17:28:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T17:28:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: mon scrub interval = 300 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T17:28:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T17:28:05.390 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-17T17:28:05.390 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-17T17:28:05.390 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-17T17:28:05.390 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2024-09-17T17:28:05.391 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2024-09-17T17:28:05.392 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.393 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.7,172.21.15.26,172.21.15.27" 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: mon client directed command retry = 5 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: mon election default strategy = 1 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay max = 1 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay probability = 0.005 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay type = osd 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject internal delays = 0.002 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject socket failures = 2500 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: osd_async_recovery_min_cost = 1 2024-09-17T17:28:05.394 INFO:teuthology.orchestra.run.smithi027.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 120 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2024-09-17T17:28:05.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug trim objects = true 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2024-09-17T17:28:05.396 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: debug monc = 20 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bluefs allocator = hybrid 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore allocator = hybrid 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore write v2 = False 2024-09-17T17:28:05.397 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore zero block detection = True 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 20 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 20 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 10 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: osd blocked scrub grace period = 3600 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug reject backfill probability = 0.3 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify cached snaps = True 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify missing on start = True 2024-09-17T17:28:05.398 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd max backfills = 3 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd max markdown count = 1000 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock override recovery settings = True 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd op queue = debug_random 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd op queue cut off = debug_random 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub min interval = 60 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: osd shutdown pgref assert = True 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2024-09-17T17:28:05.399 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-17T17:28:05.400 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T17:28:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: mon scrub interval = 300 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2024-09-17T17:28:05.402 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-09-17T17:28:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-09-17T17:28:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-09-17T17:28:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-09-17T17:28:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.7,172.21.15.26,172.21.15.27" 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: mon client directed command retry = 5 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: mon election default strategy = 1 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: ms inject delay max = 1 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: ms inject delay probability = 0.005 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: ms inject delay type = osd 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: ms inject internal delays = 0.002 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: ms inject socket failures = 2500 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: osd_async_recovery_min_cost = 1 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-09-17T17:28:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 120 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug trim objects = true 2024-09-17T17:28:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: debug monc = 20 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: bluefs allocator = hybrid 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = hybrid 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore write v2 = False 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore zero block detection = True 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 20 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 20 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-09-17T17:28:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 10 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd blocked scrub grace period = 3600 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug reject backfill probability = 0.3 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify cached snaps = True 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify missing on start = True 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd max backfills = 3 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd max markdown count = 1000 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock override recovery settings = True 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-09-17T17:28:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: osd op queue = debug_random 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: osd op queue cut off = debug_random 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub min interval = 60 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: osd shutdown pgref assert = True 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-09-17T17:28:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: mon scrub interval = 300 2024-09-17T17:28:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T17:28:05.415 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T17:28:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2024-09-17T17:28:05.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2024-09-17T17:28:05.418 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = "172.21.15.7,172.21.15.26,172.21.15.27" 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: mon client directed command retry = 5 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: mon election default strategy = 1 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay max = 1 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay probability = 0.005 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay type = osd 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject internal delays = 0.002 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject socket failures = 2500 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: osd_async_recovery_min_cost = 1 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:28:05.419 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 120 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock profile = high_recovery_ops 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2024-09-17T17:28:05.420 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug trim objects = true 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2024-09-17T17:28:05.421 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: debug monc = 20 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bluefs allocator = hybrid 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore allocator = hybrid 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore write v2 = False 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore zero block detection = True 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 20 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 20 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-17T17:28:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 10 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd blocked scrub grace period = 3600 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug reject backfill probability = 0.3 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify cached snaps = True 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify missing on start = True 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd max backfills = 3 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd max markdown count = 1000 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock override recovery settings = True 2024-09-17T17:28:05.423 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: osd op queue = debug_random 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: osd op queue cut off = debug_random 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub min interval = 60 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: osd shutdown pgref assert = True 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-09-17T17:28:05.424 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2024-09-17T17:28:05.425 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2024-09-17T17:28:05.426 INFO:teuthology.orchestra.run.smithi102.stdout: mon scrub interval = 300 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.b] 2024-09-17T17:28:05.427 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.c] 2024-09-17T17:28:05.430 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T17:28:05.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T17:28:05.486 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T17:28:05.486 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:05.486 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T17:28:05.503 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:05.503 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T17:28:05.559 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:28:05.559 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:05.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T17:28:05.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T17:28:05.636 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:28:05.636 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:05.692 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:28:05.692 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:05.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T17:28:05.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T17:28:05.735 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:28:05.735 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:05.794 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:28:05.794 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:05.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T17:28:05.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T17:28:05.836 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:28:05.837 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:05.895 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:28:05.895 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:05.895 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T17:28:05.896 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T17:28:05.937 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:28:05.937 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:05.996 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T17:28:05.996 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T17:28:05.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T17:28:06.050 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T17:28:06.053 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T17:28:06.129 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T17:28:06.132 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T17:28:06.132 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T17:28:06.132 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T17:28:06.176 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T17:28:06.189 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T17:28:06.189 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-17T17:28:06.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T17:28:06.215 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:06.215 INFO:tasks.ceph:role: osd.0 2024-09-17T17:28:06.215 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:28:06.215 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T17:28:06.280 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:06.280 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:06.280 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:06.280 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:06.280 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:06.280 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:06.281 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:06.281 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:06.281 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:06.281 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:06.313 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T17:28:06.315 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T17:28:06.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T17:28:06.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T17:28:06.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T17:28:06.520 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:06.520 INFO:tasks.ceph:role: osd.4 2024-09-17T17:28:06.520 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:28:06.521 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T17:28:06.586 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:06.586 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:06.586 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:06.586 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:06.587 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:06.587 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:06.587 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:06.587 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:06.587 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:06.587 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:06.618 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T17:28:06.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T17:28:06.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-17T17:28:06.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T17:28:06.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-17T17:28:06.829 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:06.829 INFO:tasks.ceph:role: osd.8 2024-09-17T17:28:06.829 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:28:06.829 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T17:28:06.894 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:06.895 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:06.926 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T17:28:06.928 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T17:28:06.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-09-17T17:28:06.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-17T17:28:07.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-09-17T17:28:07.135 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:07.135 INFO:tasks.ceph:role: osd.12 2024-09-17T17:28:07.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:28:07.135 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T17:28:07.200 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:07.201 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:07.201 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:07.201 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:07.201 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:07.233 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T17:28:07.235 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T17:28:07.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-09-17T17:28:07.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-09-17T17:28:07.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:07.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:07.472+0000 7fabfc0d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T17:28:07.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:07.472+0000 7fabfc0d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T17:28:07.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:07.472+0000 7fabfc0d6640 -1 bdev(0x55bbc51db800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:07.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:07.472+0000 7fabfc0d6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T17:28:10.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T17:28:10.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:11.007+0000 7f440aa0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T17:28:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:11.007+0000 7f440aa0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T17:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:11.007+0000 7f440aa0d640 -1 bdev(0x55bf21eb9800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:11.007+0000 7f440aa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T17:28:14.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T17:28:14.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:14.531+0000 7f26b7934640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-17T17:28:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:14.531+0000 7f26b7934640 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-17T17:28:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:14.531+0000 7f26b7934640 -1 bdev(0x5576479bb800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:14.531+0000 7f26b7934640 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-17T17:28:17.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-17T17:28:17.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:17.567+0000 7fd7c8816640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-09-17T17:28:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:17.567+0000 7fd7c8816640 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-09-17T17:28:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:17.567+0000 7fd7c8816640 -1 bdev(0x55cdd1ad9800 /var/lib/ceph/osd/ceph-12/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:28:17.567+0000 7fd7c8816640 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-09-17T17:28:20.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-09-17T17:28:21.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T17:28:21.053 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:21.053 INFO:tasks.ceph:role: osd.1 2024-09-17T17:28:21.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:28:21.053 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T17:28:21.122 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:21.123 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:21.155 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-09-17T17:28:21.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-17T17:28:21.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-17T17:28:21.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T17:28:21.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T17:28:21.368 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:21.368 INFO:tasks.ceph:role: osd.5 2024-09-17T17:28:21.368 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:28:21.368 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T17:28:21.433 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:21.433 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:21.433 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:21.433 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:21.434 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:21.434 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:21.434 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:21.434 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:21.434 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:21.434 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:21.466 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-09-17T17:28:21.468 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-17T17:28:21.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T17:28:21.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T17:28:21.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-17T17:28:21.674 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:21.674 INFO:tasks.ceph:role: osd.9 2024-09-17T17:28:21.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:28:21.675 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T17:28:21.740 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:21.740 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:21.741 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:21.773 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-09-17T17:28:21.775 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-17T17:28:21.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-09-17T17:28:21.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-17T17:28:21.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-09-17T17:28:21.982 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:21.982 INFO:tasks.ceph:role: osd.13 2024-09-17T17:28:21.982 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:28:21.982 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T17:28:22.047 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:22.048 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:22.049 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:22.049 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:22.080 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-09-17T17:28:22.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-17T17:28:22.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-09-17T17:28:22.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-09-17T17:28:22.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:22.333+0000 7fa03601c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T17:28:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:22.333+0000 7fa03601c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T17:28:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:22.333+0000 7fa03601c640 -1 bdev(0x560e70d75800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:22.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:22.333+0000 7fa03601c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T17:28:25.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T17:28:25.771 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:25.862+0000 7f6795a18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T17:28:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:25.863+0000 7f6795a18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T17:28:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:25.863+0000 7f6795a18640 -1 bdev(0x5568787a9800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:25.863+0000 7f6795a18640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T17:28:29.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T17:28:29.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:29.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:29.417+0000 7f3d727dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-17T17:28:29.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:29.417+0000 7f3d727dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-17T17:28:29.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:29.417+0000 7f3d727dd640 -1 bdev(0x555609dfd800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:29.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:29.418+0000 7f3d727dd640 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-17T17:28:32.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-17T17:28:32.372 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:32.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:32.463+0000 7f0eb162c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-09-17T17:28:32.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:32.463+0000 7f0eb162c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-09-17T17:28:32.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:32.463+0000 7f0eb162c640 -1 bdev(0x555b6e9ef800 /var/lib/ceph/osd/ceph-13/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:32.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-17T17:28:32.463+0000 7f0eb162c640 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-09-17T17:28:35.401 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-09-17T17:28:35.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T17:28:35.464 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:35.464 INFO:tasks.ceph:role: osd.2 2024-09-17T17:28:35.464 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:28:35.464 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T17:28:35.542 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:35.543 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:36.052 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-17T17:28:36.055 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-17T17:28:36.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-17T17:28:36.163 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T17:28:36.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T17:28:36.265 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:36.265 INFO:tasks.ceph:role: osd.6 2024-09-17T17:28:36.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:28:36.265 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T17:28:36.341 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:36.341 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:36.342 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:36.342 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:36.342 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:36.796 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-17T17:28:36.799 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-17T17:28:36.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-09-17T17:28:36.893 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T17:28:36.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-17T17:28:36.991 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:36.991 INFO:tasks.ceph:role: osd.10 2024-09-17T17:28:36.991 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:28:36.991 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T17:28:37.064 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:37.065 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:37.066 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:37.551 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-17T17:28:37.553 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-17T17:28:37.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-17T17:28:37.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-17T17:28:37.724 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-09-17T17:28:37.790 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:37.790 INFO:tasks.ceph:role: osd.14 2024-09-17T17:28:37.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:28:37.790 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T17:28:37.863 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:37.863 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:37.864 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:38.383 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-17T17:28:38.385 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-17T17:28:38.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-09-17T17:28:38.530 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-09-17T17:28:38.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:38.662+0000 7f33e1218640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T17:28:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:38.662+0000 7f33e1218640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T17:28:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:38.662+0000 7f33e1218640 -1 bdev(0x55d8c812b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:38.662+0000 7f33e1218640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T17:28:42.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T17:28:42.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:42.199+0000 7fddc46f1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T17:28:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:42.200+0000 7fddc46f1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T17:28:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:42.200+0000 7fddc46f1640 -1 bdev(0x56344939d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:42.200+0000 7fddc46f1640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T17:28:45.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T17:28:45.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:45.493+0000 7f21531fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-17T17:28:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:45.494+0000 7f21531fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-17T17:28:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:45.494+0000 7f21531fd640 -1 bdev(0x55ccd70d1800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:45.494+0000 7f21531fd640 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-17T17:28:48.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-17T17:28:48.456 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:48.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:48.548+0000 7f3f5d1dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-09-17T17:28:48.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:48.548+0000 7f3f5d1dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-09-17T17:28:48.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:48.548+0000 7f3f5d1dd640 -1 bdev(0x55b67b7bf800 /var/lib/ceph/osd/ceph-14/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:48.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-17T17:28:48.548+0000 7f3f5d1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-09-17T17:28:51.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-09-17T17:28:51.495 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T17:28:51.525 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:51.525 INFO:tasks.ceph:role: osd.3 2024-09-17T17:28:51.525 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:28:51.526 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T17:28:51.599 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:51.600 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:52.078 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-17T17:28:52.079 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-17T17:28:52.079 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-17T17:28:52.196 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T17:28:52.227 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T17:28:52.294 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:52.294 INFO:tasks.ceph:role: osd.7 2024-09-17T17:28:52.294 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:28:52.294 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:52.367 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:52.368 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:52.824 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-17T17:28:52.826 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-17T17:28:52.826 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-09-17T17:28:52.898 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T17:28:52.970 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-17T17:28:53.035 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:53.035 INFO:tasks.ceph:role: osd.11 2024-09-17T17:28:53.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:28:53.036 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T17:28:53.107 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:53.108 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:53.593 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-17T17:28:53.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-17T17:28:53.595 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-09-17T17:28:53.682 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-17T17:28:53.711 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-09-17T17:28:53.776 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-17T17:28:53.776 INFO:tasks.ceph:role: osd.15 2024-09-17T17:28:53.777 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:28:53.777 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T17:28:53.849 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T17:28:53.849 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T17:28:53.849 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T17:28:53.849 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T17:28:53.849 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T17:28:53.849 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-17T17:28:53.850 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T17:28:53.850 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T17:28:53.850 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T17:28:53.850 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T17:28:54.354 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-17T17:28:54.356 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-17T17:28:54.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-09-17T17:28:54.492 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-09-17T17:28:54.526 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:54.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:54.623+0000 7ff652a12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T17:28:54.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:54.623+0000 7ff652a12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T17:28:54.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:54.623+0000 7ff652a12640 -1 bdev(0x55c23c8e9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:54.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:54.623+0000 7ff652a12640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T17:28:57.777 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T17:28:57.807 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:28:57.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:57.895+0000 7fceb0cd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T17:28:57.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:57.895+0000 7fceb0cd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T17:28:57.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:57.895+0000 7fceb0cd8640 -1 bdev(0x55809601b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T17:28:57.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:28:57.895+0000 7fceb0cd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T17:29:01.056 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T17:29:01.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:29:01.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:01.174+0000 7f6fb5ddd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-17T17:29:01.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:01.175+0000 7f6fb5ddd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-17T17:29:01.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:01.175+0000 7f6fb5ddd640 -1 bdev(0x55bf440a7800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2024-09-17T17:29:01.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:01.175+0000 7f6fb5ddd640 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-17T17:29:04.092 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-17T17:29:04.120 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:29:04.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:04.205+0000 7fee3ba1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-09-17T17:29:04.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:04.206+0000 7fee3ba1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-09-17T17:29:04.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:04.206+0000 7fee3ba1b640 -1 bdev(0x5588e46ef800 /var/lib/ceph/osd/ceph-15/block) open stat got: (1) Operation not permitted 2024-09-17T17:29:04.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-17T17:29:04.206+0000 7fee3ba1b640 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-09-17T17:29:07.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-09-17T17:29:07.403 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T17:29:07.403 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:07.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T17:29:07.435 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:07.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T17:29:07.505 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:07.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T17:29:07.574 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:07.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-17T17:29:07.642 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:07.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-09-17T17:29:07.710 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:07.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T17:29:07.742 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:07.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T17:29:07.812 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:07.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-17T17:29:07.881 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:07.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-09-17T17:29:07.951 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:07.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T17:29:07.983 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:07.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T17:29:08.051 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:08.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-17T17:29:08.119 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:08.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-09-17T17:29:08.185 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:08.186 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T17:29:08.214 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:08.215 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T17:29:08.283 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:08.283 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T17:29:08.350 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:08.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-17T17:29:08.416 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:08.416 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-09-17T17:29:08.481 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:08.482 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T17:29:08.538 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T17:29:08.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T17:29:08.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T17:29:08.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T17:29:08.567 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-09-17T17:29:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAmvOlmgwX2AhAA+/Zg3LR9QxLAL0iR38wz4A== 2024-09-17T17:29:08.567 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-17T17:29:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAnvOlm8QQxHBAA4xBpNl+HtBSGrxqMkEclOA== 2024-09-17T17:29:08.567 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-17T17:29:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQArvOlmVKV+ABAA542nqi/WO8N/2TBiwR3+xg== 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.8] 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAuvOlm0bG4HxAAiIoUcT/e+D1wAnBj4Zmixw== 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.12] 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAxvOlmCH3fIRAA8FWwRyQ7pqXIn+f0ZxyePg== 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA2vOlmD6H/ExAAeNzaCpCu3aoHlbH/C7Umdg== 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA5vOlmrZV8MxAAgqoBb9/6aiofEWSf62lG7w== 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.9] 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA9vOlmv9XzGBAAw6VeQt9fAiI/hqKMQcLBzw== 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.13] 2024-09-17T17:29:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBAvOlmAeisGxAAmo1cLTobUEeqB+ahdSU50g== 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBGvOlm68yPJxAABSaXCNNx9YBzBRZyMJeLTw== 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBKvOlmMuIADBAAXlo5e4GmyuP6/UY+WB4gcg== 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.10] 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBNvOlmykeFHRAAN/yLcOG2hQ2lBx2mD2Mzcw== 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.14] 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBQvOlm/fnAIBAAOE5cZp80PkIAJpke/vd4pg== 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAmvOlmctypBxAAR7YeUeIuRiQ7p1n4OxtYLA== 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-17T17:29:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBWvOlmpPo9JRAAimfCdVVX7Ez/diwldGXVdQ== 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBZvOlmOn53NRAAu5n+VKNQiThIJRyK+nQZYA== 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.11] 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBdvOlmsTmHChAAtpzZoaXzSlQUqSCt/ZDZxA== 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.15] 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBgvOlmi2tgDBAAZBIqJq8vmLlyR4f6adG2Iw== 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-17T17:29:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAmvOlmnp5yChAAHg/WL2B72zBAjgZlXjup5A== 2024-09-17T17:29:08.572 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2024-09-17T17:29:08.572 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAmvOlmgwX2AhAA+/Zg3LR9QxLAL0iR38wz4A== 2024-09-17T17:29:08.572 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-09-17T17:29:08.572 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAnvOlm8QQxHBAA4xBpNl+HtBSGrxqMkEclOA== 2024-09-17T17:29:08.572 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQArvOlmVKV+ABAA542nqi/WO8N/2TBiwR3+xg== 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.8] 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAuvOlm0bG4HxAAiIoUcT/e+D1wAnBj4Zmixw== 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.12] 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAxvOlmCH3fIRAA8FWwRyQ7pqXIn+f0ZxyePg== 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA2vOlmD6H/ExAAeNzaCpCu3aoHlbH/C7Umdg== 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA5vOlmrZV8MxAAgqoBb9/6aiofEWSf62lG7w== 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.9] 2024-09-17T17:29:08.573 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA9vOlmv9XzGBAAw6VeQt9fAiI/hqKMQcLBzw== 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.13] 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBAvOlmAeisGxAAmo1cLTobUEeqB+ahdSU50g== 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBGvOlm68yPJxAABSaXCNNx9YBzBRZyMJeLTw== 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBKvOlmMuIADBAAXlo5e4GmyuP6/UY+WB4gcg== 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.10] 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBNvOlmykeFHRAAN/yLcOG2hQ2lBx2mD2Mzcw== 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.14] 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBQvOlm/fnAIBAAOE5cZp80PkIAJpke/vd4pg== 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-09-17T17:29:08.574 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAmvOlmctypBxAAR7YeUeIuRiQ7p1n4OxtYLA== 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBWvOlmpPo9JRAAimfCdVVX7Ez/diwldGXVdQ== 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBZvOlmOn53NRAAu5n+VKNQiThIJRyK+nQZYA== 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.11] 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBdvOlmsTmHChAAtpzZoaXzSlQUqSCt/ZDZxA== 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.15] 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBgvOlmi2tgDBAAZBIqJq8vmLlyR4f6adG2Iw== 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAmvOlmnp5yChAAHg/WL2B72zBAjgZlXjup5A== 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.y] 2024-09-17T17:29:08.575 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAmvOlmgwX2AhAA+/Zg3LR9QxLAL0iR38wz4A== 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAnvOlm8QQxHBAA4xBpNl+HtBSGrxqMkEclOA== 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQArvOlmVKV+ABAA542nqi/WO8N/2TBiwR3+xg== 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.8] 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAuvOlm0bG4HxAAiIoUcT/e+D1wAnBj4Zmixw== 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.12] 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAxvOlmCH3fIRAA8FWwRyQ7pqXIn+f0ZxyePg== 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQA2vOlmD6H/ExAAeNzaCpCu3aoHlbH/C7Umdg== 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2024-09-17T17:29:08.576 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQA5vOlmrZV8MxAAgqoBb9/6aiofEWSf62lG7w== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.9] 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQA9vOlmv9XzGBAAw6VeQt9fAiI/hqKMQcLBzw== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.13] 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBAvOlmAeisGxAAmo1cLTobUEeqB+ahdSU50g== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGvOlm68yPJxAABSaXCNNx9YBzBRZyMJeLTw== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBKvOlmMuIADBAAXlo5e4GmyuP6/UY+WB4gcg== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.10] 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBNvOlmykeFHRAAN/yLcOG2hQ2lBx2mD2Mzcw== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.14] 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBQvOlm/fnAIBAAOE5cZp80PkIAJpke/vd4pg== 2024-09-17T17:29:08.577 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAmvOlmctypBxAAR7YeUeIuRiQ7p1n4OxtYLA== 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBWvOlmpPo9JRAAimfCdVVX7Ez/diwldGXVdQ== 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBZvOlmOn53NRAAu5n+VKNQiThIJRyK+nQZYA== 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.11] 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBdvOlmsTmHChAAtpzZoaXzSlQUqSCt/ZDZxA== 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.15] 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBgvOlmi2tgDBAAZBIqJq8vmLlyR4f6adG2Iw== 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-17T17:29:08.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAmvOlmnp5yChAAHg/WL2B72zBAjgZlXjup5A== 2024-09-17T17:29:08.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T17:29:08.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T17:29:08.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T17:29:08.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:08.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.084 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.210 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T17:29:09.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T17:29:09.383 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T17:29:09.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.496 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T17:29:09.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T17:29:09.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T17:29:09.668 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T17:29:09.724 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T17:29:09.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T17:29:09.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T17:29:09.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T17:29:09.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T17:29:09.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T17:29:09.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T17:29:10.020 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T17:29:10.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T17:29:10.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T17:29:10.172 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:29:10.175 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:29:10.177 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T17:29:10.228 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T17:29:10.228 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T17:29:10.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T17:29:10.231 INFO:tasks.ceph.mon.a:Started 2024-09-17T17:29:10.231 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T17:29:10.231 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T17:29:10.233 INFO:tasks.ceph.mon.b:Started 2024-09-17T17:29:10.234 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T17:29:10.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T17:29:10.270 INFO:tasks.ceph.mon.c:Started 2024-09-17T17:29:10.271 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T17:29:10.271 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T17:29:10.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T17:29:10.273 INFO:tasks.ceph.mgr.y:Started 2024-09-17T17:29:10.273 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T17:29:10.274 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T17:29:10.276 INFO:tasks.ceph.mgr.x:Started 2024-09-17T17:29:10.276 DEBUG:tasks.ceph:set 0 configs 2024-09-17T17:29:10.277 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T17:29:10.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T17:29:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-17T17:29:15.678 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T17:29:15.678 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T17:29:15.678 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:15.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T17:29:15.705 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:15.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T17:29:15.771 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:15.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-17T17:29:15.837 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T17:29:15.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-09-17T17:29:15.903 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:15.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T17:29:15.934 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:15.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T17:29:16.005 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:16.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-17T17:29:16.074 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-17T17:29:16.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-09-17T17:29:16.144 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:16.144 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T17:29:16.175 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:16.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T17:29:16.244 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:16.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-17T17:29:16.313 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-17T17:29:16.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-09-17T17:29:16.381 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:16.381 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T17:29:16.406 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:16.406 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T17:29:16.470 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:16.470 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-17T17:29:16.534 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-17T17:29:16.534 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-09-17T17:29:16.599 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 72ee94e9-57bf-46b3-9fb9-0d05515fe1e1 0 2024-09-17T17:29:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:0 2024-09-17T17:29:16.831 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new e9764092-a458-4b6d-8bee-658f54f000cc 1 2024-09-17T17:29:17.012 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-09-17T17:29:17.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new f1142f7d-9945-4daa-8f6a-6fd2d3090b5f 2 2024-09-17T17:29:17.203 INFO:teuthology.orchestra.run.smithi102.stdout:2 2024-09-17T17:29:17.215 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new daa4b149-c71a-4cba-a710-89aa101d7d96 3 2024-09-17T17:29:17.395 INFO:teuthology.orchestra.run.smithi102.stdout:3 2024-09-17T17:29:17.406 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 55e8e034-493b-4066-a42b-f4d82294f3f8 4 2024-09-17T17:29:17.586 INFO:teuthology.orchestra.run.smithi102.stdout:4 2024-09-17T17:29:17.597 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 38a09763-d370-447a-b939-bad8a11c5e01 5 2024-09-17T17:29:17.780 INFO:teuthology.orchestra.run.smithi102.stdout:5 2024-09-17T17:29:17.791 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 0ae069f9-8831-4bfc-9928-2100b73a839d 6 2024-09-17T17:29:17.970 INFO:teuthology.orchestra.run.smithi102.stdout:6 2024-09-17T17:29:17.981 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b 7 2024-09-17T17:29:18.163 INFO:teuthology.orchestra.run.smithi102.stdout:7 2024-09-17T17:29:18.174 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 3c4daae0-6d93-450f-8600-b29362bd9f77 8 2024-09-17T17:29:18.353 INFO:teuthology.orchestra.run.smithi102.stdout:8 2024-09-17T17:29:18.365 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 2846b3c6-06d6-414f-b4c9-b16d94007737 9 2024-09-17T17:29:18.545 INFO:teuthology.orchestra.run.smithi102.stdout:9 2024-09-17T17:29:18.556 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 495d7d7c-c222-4a46-9685-2a2a7cc6fc51 10 2024-09-17T17:29:18.736 INFO:teuthology.orchestra.run.smithi102.stdout:10 2024-09-17T17:29:18.747 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new cf76d383-65cd-48c3-a54c-cd83ffbc9898 11 2024-09-17T17:29:18.927 INFO:teuthology.orchestra.run.smithi102.stdout:11 2024-09-17T17:29:18.938 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 54294878-ea1e-4845-a0e6-9be026f521fa 12 2024-09-17T17:29:19.120 INFO:teuthology.orchestra.run.smithi102.stdout:12 2024-09-17T17:29:19.131 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 9acf83a5-75db-4874-8f3f-7421cc55e2fd 13 2024-09-17T17:29:19.312 INFO:teuthology.orchestra.run.smithi102.stdout:13 2024-09-17T17:29:19.323 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 16c02ebe-a418-4f87-902d-b5dac864f139 14 2024-09-17T17:29:19.503 INFO:teuthology.orchestra.run.smithi102.stdout:14 2024-09-17T17:29:19.514 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa 15 2024-09-17T17:29:19.696 INFO:teuthology.orchestra.run.smithi102.stdout:15 2024-09-17T17:29:19.707 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T17:29:19.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T17:29:19.710 INFO:tasks.ceph.osd.0:Started 2024-09-17T17:29:19.710 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T17:29:19.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T17:29:19.712 INFO:tasks.ceph.osd.4:Started 2024-09-17T17:29:19.712 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-17T17:29:19.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-17T17:29:19.714 INFO:tasks.ceph.osd.8:Started 2024-09-17T17:29:19.715 INFO:tasks.ceph.osd.12:Restarting daemon 2024-09-17T17:29:19.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-09-17T17:29:19.717 INFO:tasks.ceph.osd.12:Started 2024-09-17T17:29:19.717 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T17:29:19.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T17:29:19.720 INFO:tasks.ceph.osd.1:Started 2024-09-17T17:29:19.720 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T17:29:19.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T17:29:19.722 INFO:tasks.ceph.osd.5:Started 2024-09-17T17:29:19.723 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-17T17:29:19.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-17T17:29:19.725 INFO:tasks.ceph.osd.9:Started 2024-09-17T17:29:19.725 INFO:tasks.ceph.osd.13:Restarting daemon 2024-09-17T17:29:19.725 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-09-17T17:29:19.727 INFO:tasks.ceph.osd.13:Started 2024-09-17T17:29:19.727 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T17:29:19.728 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T17:29:19.730 INFO:tasks.ceph.osd.2:Started 2024-09-17T17:29:19.730 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T17:29:19.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T17:29:19.733 INFO:tasks.ceph.osd.6:Started 2024-09-17T17:29:19.733 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-17T17:29:19.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-17T17:29:19.735 INFO:tasks.ceph.osd.10:Started 2024-09-17T17:29:19.735 INFO:tasks.ceph.osd.14:Restarting daemon 2024-09-17T17:29:19.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-09-17T17:29:19.738 INFO:tasks.ceph.osd.14:Started 2024-09-17T17:29:19.738 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T17:29:19.738 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T17:29:19.740 INFO:tasks.ceph.osd.3:Started 2024-09-17T17:29:19.740 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T17:29:19.740 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T17:29:19.742 INFO:tasks.ceph.osd.7:Started 2024-09-17T17:29:19.742 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-17T17:29:19.743 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-17T17:29:19.745 INFO:tasks.ceph.osd.11:Started 2024-09-17T17:29:19.745 INFO:tasks.ceph.osd.15:Restarting daemon 2024-09-17T17:29:19.745 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-09-17T17:29:19.747 INFO:tasks.ceph.osd.15:Started 2024-09-17T17:29:19.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:29:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:19.953 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:29:19.693959+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T17:29:19.693959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:29:19.964 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T17:29:19.964 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T17:29:20.696 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:20.695+0000 7f1b67c0d640 -1 Falling back to public interface 2024-09-17T17:29:20.705 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:20.703+0000 7f09c88a5640 -1 Falling back to public interface 2024-09-17T17:29:20.880 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-17T17:29:20.878+0000 7fb00322f640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T17:29:20.883 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-17T17:29:20.882+0000 7fb00322f640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T17:29:20.930 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:20.927+0000 7f1c8a81e640 -1 Falling back to public interface 2024-09-17T17:29:20.935 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:20.933+0000 7fe65fc10640 -1 Falling back to public interface 2024-09-17T17:29:20.961 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:20.959+0000 7fdc74902640 -1 Falling back to public interface 2024-09-17T17:29:21.165 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:21.163+0000 7f9fd980d640 -1 Falling back to public interface 2024-09-17T17:29:21.175 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:21.173+0000 7f8384ea5640 -1 Falling back to public interface 2024-09-17T17:29:21.183 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:21.181+0000 7fee94833640 -1 Falling back to public interface 2024-09-17T17:29:21.194 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:21.192+0000 7f9b8b934640 -1 Falling back to public interface 2024-09-17T17:29:21.201 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:21.199+0000 7fafb6b17640 -1 Falling back to public interface 2024-09-17T17:29:21.209 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:21.207+0000 7fa051945640 -1 Falling back to public interface 2024-09-17T17:29:21.240 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:21.237+0000 7fb9afa11640 -1 Falling back to public interface 2024-09-17T17:29:21.442 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:21.440+0000 7f2f7fcae640 -1 Falling back to public interface 2024-09-17T17:29:21.456 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:21.454+0000 7f3e1afcc640 -1 Falling back to public interface 2024-09-17T17:29:21.465 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:21.463+0000 7f4e6a60b640 -1 Falling back to public interface 2024-09-17T17:29:21.484 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:21.481+0000 7fb1b640d640 -1 Falling back to public interface 2024-09-17T17:29:21.575 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T17:29:21.888 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-17T17:29:21.888 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":18,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:29:19.693959+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T17:29:19.693959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:29:21.900 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-09-17T17:29:24.480 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:24.477+0000 7fe65fc10640 -1 osd.1 0 log_to_monitors true 2024-09-17T17:29:24.504 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:24.502+0000 7f1b67c0d640 -1 osd.7 0 log_to_monitors true 2024-09-17T17:29:24.548 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:24.546+0000 7fdc74902640 -1 osd.2 0 log_to_monitors true 2024-09-17T17:29:24.662 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:24.660+0000 7f9fd980d640 -1 osd.5 0 log_to_monitors true 2024-09-17T17:29:24.705 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:24.703+0000 7f1c8a81e640 -1 osd.9 0 log_to_monitors true 2024-09-17T17:29:24.738 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:24.736+0000 7f8384ea5640 -1 osd.4 0 log_to_monitors true 2024-09-17T17:29:24.749 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:24.747+0000 7f09c88a5640 -1 osd.13 0 log_to_monitors true 2024-09-17T17:29:25.000 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:24.998+0000 7fafb6b17640 -1 osd.0 0 log_to_monitors true 2024-09-17T17:29:25.027 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:25.025+0000 7fa051945640 -1 osd.12 0 log_to_monitors true 2024-09-17T17:29:25.209 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:25.207+0000 7f2f7fcae640 -1 osd.15 0 log_to_monitors true 2024-09-17T17:29:25.255 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:25.253+0000 7f9b8b934640 -1 osd.3 0 log_to_monitors true 2024-09-17T17:29:25.284 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:25.282+0000 7fb9afa11640 -1 osd.10 0 log_to_monitors true 2024-09-17T17:29:25.433 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:25.431+0000 7fee94833640 -1 osd.8 0 log_to_monitors true 2024-09-17T17:29:25.512 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:25.509+0000 7fb1b640d640 -1 osd.14 0 log_to_monitors true 2024-09-17T17:29:25.556 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:25.554+0000 7f4e6a60b640 -1 osd.11 0 log_to_monitors true 2024-09-17T17:29:25.735 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:25.732+0000 7f3e1afcc640 -1 osd.6 0 log_to_monitors true 2024-09-17T17:29:26.154 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:26.152+0000 7fe65b4fe640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T17:29:26.166 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:26.164+0000 7fe65331d640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.320 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:26.318+0000 7f1b63592640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T17:29:26.329 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:26.328+0000 7f1b5b3b1640 -1 osd.7 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.560 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:26.558+0000 7f09c4834640 -1 osd.13 0 waiting for initial osdmap 2024-09-17T17:29:26.564 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:26.562+0000 7f2f7bc3d640 -1 osd.15 0 waiting for initial osdmap 2024-09-17T17:29:26.565 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:26.562+0000 7fdc70891640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T17:29:26.565 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:26.562+0000 7fb9ab31e640 -1 osd.10 0 waiting for initial osdmap 2024-09-17T17:29:26.565 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:26.562+0000 7fb1b1e4b640 -1 osd.14 0 waiting for initial osdmap 2024-09-17T17:29:26.566 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:26.564+0000 7f9b878c3640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T17:29:26.568 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:26.567+0000 7fafb2aa6640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T17:29:26.571 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:26.569+0000 7f9b836ea640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.571 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:26.570+0000 7f09c065b640 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.579 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:26.578+0000 7f2f77a64640 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.580 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:26.578+0000 7fb1adc72640 -1 osd.14 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.581 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:26.579+0000 7fb9a7145640 -1 osd.10 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.582 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:26.580+0000 7fdc6c6b8640 -1 osd.2 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:26.622 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:26.620+0000 7fafae8cd640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:27.426 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:27.423+0000 7f1c86a20640 -1 osd.9 0 waiting for initial osdmap 2024-09-17T17:29:27.432 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:27.430+0000 7f9fd521c640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T17:29:27.442 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:27.440+0000 7f1c7e02b640 -1 osd.9 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:27.448 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:27.446+0000 7f9fcd03b640 -1 osd.5 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:27.671 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:27.669+0000 7f8380e34640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T17:29:27.683 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:27.681+0000 7f8378c53640 -1 osd.4 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:27.707 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:27.705+0000 7fee907c2640 -1 osd.8 0 waiting for initial osdmap 2024-09-17T17:29:27.716 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:27.714+0000 7fa04d8d4640 -1 osd.12 0 waiting for initial osdmap 2024-09-17T17:29:27.725 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:27.723+0000 7fee885e1640 -1 osd.8 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:27.733 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:27.731+0000 7fa0456f3640 -1 osd.12 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:28.360 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:28.359+0000 7f4e65fd9640 -1 osd.11 0 waiting for initial osdmap 2024-09-17T17:29:28.376 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:28.374+0000 7f4e5ddf8640 -1 osd.11 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:28.411 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:28.409+0000 7f3e16f5b640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T17:29:28.422 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:28.420+0000 7f3e0ed7a640 -1 osd.6 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:29:29.509 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T17:29:29.825 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-17T17:29:29.825 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":24,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:29:29.560178+0000","last_up_change":"2024-09-17T17:29:28.558396+0000","last_in_change":"2024-09-17T17:29:19.693959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6809","nonce":1273991967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6811","nonce":1273991967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6815","nonce":1273991967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6813","nonce":1273991967}]},"public_addr":"172.21.15.7:6809/1273991967","cluster_addr":"172.21.15.7:6811/1273991967","heartbeat_back_addr":"172.21.15.7:6815/1273991967","heartbeat_front_addr":"172.21.15.7:6813/1273991967","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.950873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.459602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.560618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.280057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.776579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.712128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.750496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.533175+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.401768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.738806+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.295563+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.519592+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.989375+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.757133+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.479198+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.193057+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:29:29.836 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-09-17T17:29:29.836 INFO:tasks.ceph:Creating RBD pool 2024-09-17T17:29:29.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T17:29:30.573 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2024-09-17T17:29:30.588 DEBUG:teuthology.orchestra.run.smithi007:> rbd --cluster ceph pool init rbd 2024-09-17T17:29:33.600 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T17:29:33.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T17:29:33.605 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T17:29:33.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:33.934 INFO:teuthology.orchestra.run.smithi007.stdout:[{"version":10,"timestamp":"2024-09-17T17:29:28.411531+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"8269.917310"}]}] 2024-09-17T17:29:33.934 INFO:tasks.ceph_manager:config epoch is 10 2024-09-17T17:29:33.934 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T17:29:33.934 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T17:29:33.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T17:29:34.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":363254264},{"type":"v1","addr":"172.21.15.102:6801","nonce":363254264}]},"active_addr":"172.21.15.102:6801/363254264","active_change":"2024-09-17T17:29:19.861620+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3289267916}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3387577180}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":173023012}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3990296936}]}]} 2024-09-17T17:29:34.308 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T17:29:34.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T17:29:34.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:29:34.616 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:34.616 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:29:33.585633+0000","last_up_change":"2024-09-17T17:29:28.558396+0000","last_in_change":"2024-09-17T17:29:19.693959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T17:29:30.150043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6809","nonce":1273991967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6811","nonce":1273991967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6815","nonce":1273991967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6813","nonce":1273991967}]},"public_addr":"172.21.15.7:6809/1273991967","cluster_addr":"172.21.15.7:6811/1273991967","heartbeat_back_addr":"172.21.15.7:6815/1273991967","heartbeat_front_addr":"172.21.15.7:6813/1273991967","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.950873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.459602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.560618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.280057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.776579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.712128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.750496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.533175+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.401768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.738806+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.295563+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.519592+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.989375+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.757133+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.479198+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.193057+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:29:34.629 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T17:29:34.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:29:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:34.937 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:29:33.585633+0000","last_up_change":"2024-09-17T17:29:28.558396+0000","last_in_change":"2024-09-17T17:29:19.693959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T17:29:30.150043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6809","nonce":1273991967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6811","nonce":1273991967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6815","nonce":1273991967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6813","nonce":1273991967}]},"public_addr":"172.21.15.7:6809/1273991967","cluster_addr":"172.21.15.7:6811/1273991967","heartbeat_back_addr":"172.21.15.7:6815/1273991967","heartbeat_front_addr":"172.21.15.7:6813/1273991967","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.950873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.459602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.560618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.280057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.776579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.712128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.750496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.533175+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.401768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.738806+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.295563+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.519592+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.989375+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.757133+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.479198+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:26.193057+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:29:34.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T17:29:34.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T17:29:34.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T17:29:34.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T17:29:34.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T17:29:34.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T17:29:34.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T17:29:34.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T17:29:34.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-17T17:29:34.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-17T17:29:34.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-17T17:29:34.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-17T17:29:34.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-17T17:29:34.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-17T17:29:34.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-17T17:29:34.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-17T17:29:35.386 INFO:teuthology.orchestra.run.smithi007.stdout:94489280517 2024-09-17T17:29:35.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T17:29:35.401 INFO:teuthology.orchestra.run.smithi007.stdout:94489280517 2024-09-17T17:29:35.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T17:29:35.416 INFO:teuthology.orchestra.run.smithi007.stdout:98784247812 2024-09-17T17:29:35.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T17:29:35.429 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:35.429 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:35.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-17T17:29:35.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T17:29:35.432 INFO:teuthology.orchestra.run.smithi007.stdout:94489280517 2024-09-17T17:29:35.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T17:29:35.453 INFO:teuthology.orchestra.run.smithi007.stdout:90194313221 2024-09-17T17:29:35.453 INFO:teuthology.orchestra.run.smithi007.stdout:98784247813 2024-09-17T17:29:35.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T17:29:35.454 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:35.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-17T17:29:35.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-17T17:29:35.457 INFO:teuthology.orchestra.run.smithi007.stdout:98784247812 2024-09-17T17:29:35.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-17T17:29:35.471 INFO:teuthology.orchestra.run.smithi007.stdout:98784247812 2024-09-17T17:29:35.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-17T17:29:35.483 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:35.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-17T17:29:35.484 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:35.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-17T17:29:35.495 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:35.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-17T17:29:35.510 INFO:teuthology.orchestra.run.smithi007.stdout:98784247813 2024-09-17T17:29:35.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T17:29:35.554 INFO:teuthology.orchestra.run.smithi007.stdout:90194313220 2024-09-17T17:29:35.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T17:29:36.323 INFO:teuthology.orchestra.run.smithi007.stdout:94489280515 2024-09-17T17:29:36.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.2 2024-09-17T17:29:36.520 INFO:teuthology.orchestra.run.smithi007.stdout:90194313220 2024-09-17T17:29:36.524 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.534 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.7 2024-09-17T17:29:36.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.5 2024-09-17T17:29:36.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.0 2024-09-17T17:29:36.621 INFO:teuthology.orchestra.run.smithi007.stdout:98784247812 2024-09-17T17:29:36.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.6 2024-09-17T17:29:36.666 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.685 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.689 INFO:teuthology.orchestra.run.smithi007.stdout:98784247813 2024-09-17T17:29:36.689 INFO:teuthology.orchestra.run.smithi007.stdout:98784247812 2024-09-17T17:29:36.692 INFO:teuthology.orchestra.run.smithi007.stdout:90194313220 2024-09-17T17:29:36.709 INFO:teuthology.orchestra.run.smithi007.stdout:98784247812 2024-09-17T17:29:36.711 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.714 INFO:teuthology.orchestra.run.smithi007.stdout:94489280517 2024-09-17T17:29:36.716 INFO:teuthology.orchestra.run.smithi007.stdout:98784247813 2024-09-17T17:29:36.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.8 2024-09-17T17:29:36.717 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.15 2024-09-17T17:29:36.720 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.1 2024-09-17T17:29:36.723 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.11 2024-09-17T17:29:36.724 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.727 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.12 2024-09-17T17:29:36.737 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.13 2024-09-17T17:29:36.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.3 2024-09-17T17:29:36.738 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.738 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.4 2024-09-17T17:29:36.739 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.14 2024-09-17T17:29:36.744 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.760 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.9 2024-09-17T17:29:36.776 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:36.831 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:36.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.10 2024-09-17T17:29:36.844 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:37.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T17:29:37.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T17:29:37.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T17:29:37.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T17:29:37.673 INFO:teuthology.orchestra.run.smithi007.stdout:94489280516 2024-09-17T17:29:37.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.2 2024-09-17T17:29:37.693 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:37.873 INFO:teuthology.orchestra.run.smithi007.stdout:90194313221 2024-09-17T17:29:37.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.7 2024-09-17T17:29:37.886 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:37.914 INFO:teuthology.orchestra.run.smithi007.stdout:94489280517 2024-09-17T17:29:37.919 INFO:teuthology.orchestra.run.smithi007.stdout:94489280517 2024-09-17T17:29:37.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.0 2024-09-17T17:29:37.928 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:37.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.5 2024-09-17T17:29:37.933 DEBUG:teuthology.parallel:result is None 2024-09-17T17:29:37.933 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T17:29:37.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:29:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:38.277 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:29:38.288 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-17T17:29:37.868950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":463168,"kb_used_data":5704,"kb_used_omap":24,"kb_used_meta":457063,"kb_avail":1509486272,"statfs":{"total":1546188226560,"available":1545713942528,"internally_reserved":0,"allocated":5840896,"data_stored":2969692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":468032591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277772"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.269263+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:29:34.269263+0000","last_peered":"2024-09-17T17:29:34.269263+0000","last_clean":"2024-09-17T17:29:34.269263+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:29:34.269263+0000","last_undegraded":"2024-09-17T17:29:34.269263+0000","last_fullsized":"2024-09-17T17:29:34.269263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592608+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:29:33.592608+0000","last_peered":"2024-09-17T17:29:33.592608+0000","last_clean":"2024-09-17T17:29:33.592608+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:29:33.592608+0000","last_undegraded":"2024-09-17T17:29:33.592608+0000","last_fullsized":"2024-09-17T17:29:33.592608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.236756+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:29:34.236756+0000","last_peered":"2024-09-17T17:29:34.236756+0000","last_clean":"2024-09-17T17:29:34.236756+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:29:34.236756+0000","last_undegraded":"2024-09-17T17:29:34.236756+0000","last_fullsized":"2024-09-17T17:29:34.236756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.506057+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:29:34.506057+0000","last_peered":"2024-09-17T17:29:34.506057+0000","last_clean":"2024-09-17T17:29:34.506057+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:29:34.506057+0000","last_undegraded":"2024-09-17T17:29:34.506057+0000","last_fullsized":"2024-09-17T17:29:34.506057+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592515+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:29:33.592515+0000","last_peered":"2024-09-17T17:29:33.592515+0000","last_clean":"2024-09-17T17:29:33.592515+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:29:33.592515+0000","last_undegraded":"2024-09-17T17:29:33.592515+0000","last_fullsized":"2024-09-17T17:29:33.592515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615384+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:29:33.615384+0000","last_peered":"2024-09-17T17:29:33.615384+0000","last_clean":"2024-09-17T17:29:33.615384+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:29:33.615384+0000","last_undegraded":"2024-09-17T17:29:33.615384+0000","last_fullsized":"2024-09-17T17:29:33.615384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592264+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:29:33.592264+0000","last_peered":"2024-09-17T17:29:33.592264+0000","last_clean":"2024-09-17T17:29:33.592264+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:29:33.592264+0000","last_undegraded":"2024-09-17T17:29:33.592264+0000","last_fullsized":"2024-09-17T17:29:33.592264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615439+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:29:33.615439+0000","last_peered":"2024-09-17T17:29:33.615439+0000","last_clean":"2024-09-17T17:29:33.615439+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:29:33.615439+0000","last_undegraded":"2024-09-17T17:29:33.615439+0000","last_fullsized":"2024-09-17T17:29:33.615439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"25'112","reported_seq":155,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.907788+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:29:34.907788+0000","last_peered":"2024-09-17T17:29:34.907788+0000","last_clean":"2024-09-17T17:29:34.907788+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:29:34.907788+0000","last_undegraded":"2024-09-17T17:29:34.907788+0000","last_fullsized":"2024-09-17T17:29:34.907788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,7,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":649012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,7,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,7,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":639466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:29:38.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:29:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:29:38.605 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-17T17:29:37.868950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":463168,"kb_used_data":5704,"kb_used_omap":24,"kb_used_meta":457063,"kb_avail":1509486272,"statfs":{"total":1546188226560,"available":1545713942528,"internally_reserved":0,"allocated":5840896,"data_stored":2969692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":468032591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277772"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.269263+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:29:34.269263+0000","last_peered":"2024-09-17T17:29:34.269263+0000","last_clean":"2024-09-17T17:29:34.269263+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:29:34.269263+0000","last_undegraded":"2024-09-17T17:29:34.269263+0000","last_fullsized":"2024-09-17T17:29:34.269263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592608+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:29:33.592608+0000","last_peered":"2024-09-17T17:29:33.592608+0000","last_clean":"2024-09-17T17:29:33.592608+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:29:33.592608+0000","last_undegraded":"2024-09-17T17:29:33.592608+0000","last_fullsized":"2024-09-17T17:29:33.592608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.236756+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:29:34.236756+0000","last_peered":"2024-09-17T17:29:34.236756+0000","last_clean":"2024-09-17T17:29:34.236756+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:29:34.236756+0000","last_undegraded":"2024-09-17T17:29:34.236756+0000","last_fullsized":"2024-09-17T17:29:34.236756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.506057+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:29:34.506057+0000","last_peered":"2024-09-17T17:29:34.506057+0000","last_clean":"2024-09-17T17:29:34.506057+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:29:34.506057+0000","last_undegraded":"2024-09-17T17:29:34.506057+0000","last_fullsized":"2024-09-17T17:29:34.506057+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592515+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:29:33.592515+0000","last_peered":"2024-09-17T17:29:33.592515+0000","last_clean":"2024-09-17T17:29:33.592515+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:29:33.592515+0000","last_undegraded":"2024-09-17T17:29:33.592515+0000","last_fullsized":"2024-09-17T17:29:33.592515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615384+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:29:33.615384+0000","last_peered":"2024-09-17T17:29:33.615384+0000","last_clean":"2024-09-17T17:29:33.615384+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:29:33.615384+0000","last_undegraded":"2024-09-17T17:29:33.615384+0000","last_fullsized":"2024-09-17T17:29:33.615384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592264+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:29:33.592264+0000","last_peered":"2024-09-17T17:29:33.592264+0000","last_clean":"2024-09-17T17:29:33.592264+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:29:33.592264+0000","last_undegraded":"2024-09-17T17:29:33.592264+0000","last_fullsized":"2024-09-17T17:29:33.592264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615439+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:29:33.615439+0000","last_peered":"2024-09-17T17:29:33.615439+0000","last_clean":"2024-09-17T17:29:33.615439+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:29:33.615439+0000","last_undegraded":"2024-09-17T17:29:33.615439+0000","last_fullsized":"2024-09-17T17:29:33.615439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"25'112","reported_seq":155,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.907788+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:29:34.907788+0000","last_peered":"2024-09-17T17:29:34.907788+0000","last_clean":"2024-09-17T17:29:34.907788+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:29:34.907788+0000","last_undegraded":"2024-09-17T17:29:34.907788+0000","last_fullsized":"2024-09-17T17:29:34.907788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,7,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":649012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,7,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,7,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":639466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:29:38.606 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T17:29:38.606 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T17:29:38.606 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T17:29:38.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T17:29:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:38.949 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T17:29:38.962 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T17:29:38.962 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T17:29:38.973 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T17:29:38.974 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath10-testing-2024-09-17-1348'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath10-testing-2024-09-17-1348', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}} 2024-09-17T17:29:38.974 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T17:29:38.974 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T17:29:38.974 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T17:29:38.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:29:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:29:39.288 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-17T17:29:37.868950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":463168,"kb_used_data":5704,"kb_used_omap":24,"kb_used_meta":457063,"kb_avail":1509486272,"statfs":{"total":1546188226560,"available":1545713942528,"internally_reserved":0,"allocated":5840896,"data_stored":2969692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":468032591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277772"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.269263+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:29:34.269263+0000","last_peered":"2024-09-17T17:29:34.269263+0000","last_clean":"2024-09-17T17:29:34.269263+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:29:34.269263+0000","last_undegraded":"2024-09-17T17:29:34.269263+0000","last_fullsized":"2024-09-17T17:29:34.269263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592608+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:29:33.592608+0000","last_peered":"2024-09-17T17:29:33.592608+0000","last_clean":"2024-09-17T17:29:33.592608+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:29:33.592608+0000","last_undegraded":"2024-09-17T17:29:33.592608+0000","last_fullsized":"2024-09-17T17:29:33.592608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.236756+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:29:34.236756+0000","last_peered":"2024-09-17T17:29:34.236756+0000","last_clean":"2024-09-17T17:29:34.236756+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:29:34.236756+0000","last_undegraded":"2024-09-17T17:29:34.236756+0000","last_fullsized":"2024-09-17T17:29:34.236756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.506057+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:29:34.506057+0000","last_peered":"2024-09-17T17:29:34.506057+0000","last_clean":"2024-09-17T17:29:34.506057+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:29:34.506057+0000","last_undegraded":"2024-09-17T17:29:34.506057+0000","last_fullsized":"2024-09-17T17:29:34.506057+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592515+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:29:33.592515+0000","last_peered":"2024-09-17T17:29:33.592515+0000","last_clean":"2024-09-17T17:29:33.592515+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:29:33.592515+0000","last_undegraded":"2024-09-17T17:29:33.592515+0000","last_fullsized":"2024-09-17T17:29:33.592515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615384+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:29:33.615384+0000","last_peered":"2024-09-17T17:29:33.615384+0000","last_clean":"2024-09-17T17:29:33.615384+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:29:33.615384+0000","last_undegraded":"2024-09-17T17:29:33.615384+0000","last_fullsized":"2024-09-17T17:29:33.615384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592264+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:29:33.592264+0000","last_peered":"2024-09-17T17:29:33.592264+0000","last_clean":"2024-09-17T17:29:33.592264+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:29:33.592264+0000","last_undegraded":"2024-09-17T17:29:33.592264+0000","last_fullsized":"2024-09-17T17:29:33.592264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615439+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:29:33.615439+0000","last_peered":"2024-09-17T17:29:33.615439+0000","last_clean":"2024-09-17T17:29:33.615439+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:29:33.615439+0000","last_undegraded":"2024-09-17T17:29:33.615439+0000","last_fullsized":"2024-09-17T17:29:33.615439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"25'112","reported_seq":155,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.907788+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:29:34.907788+0000","last_peered":"2024-09-17T17:29:34.907788+0000","last_clean":"2024-09-17T17:29:34.907788+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:29:34.907788+0000","last_undegraded":"2024-09-17T17:29:34.907788+0000","last_fullsized":"2024-09-17T17:29:34.907788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,7,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":649012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,7,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,7,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":639466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:29:39.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:29:39.591 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:29:39.592 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:29:39.604 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-17T17:29:37.868950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":463168,"kb_used_data":5704,"kb_used_omap":24,"kb_used_meta":457063,"kb_avail":1509486272,"statfs":{"total":1546188226560,"available":1545713942528,"internally_reserved":0,"allocated":5840896,"data_stored":2969692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":468032591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277772"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.269263+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:29:34.269263+0000","last_peered":"2024-09-17T17:29:34.269263+0000","last_clean":"2024-09-17T17:29:34.269263+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:29:34.269263+0000","last_undegraded":"2024-09-17T17:29:34.269263+0000","last_fullsized":"2024-09-17T17:29:34.269263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592608+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:29:33.592608+0000","last_peered":"2024-09-17T17:29:33.592608+0000","last_clean":"2024-09-17T17:29:33.592608+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:29:33.592608+0000","last_undegraded":"2024-09-17T17:29:33.592608+0000","last_fullsized":"2024-09-17T17:29:33.592608+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.236756+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:29:34.236756+0000","last_peered":"2024-09-17T17:29:34.236756+0000","last_clean":"2024-09-17T17:29:34.236756+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:29:34.236756+0000","last_undegraded":"2024-09-17T17:29:34.236756+0000","last_fullsized":"2024-09-17T17:29:34.236756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.506057+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:29:34.506057+0000","last_peered":"2024-09-17T17:29:34.506057+0000","last_clean":"2024-09-17T17:29:34.506057+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:29:34.506057+0000","last_undegraded":"2024-09-17T17:29:34.506057+0000","last_fullsized":"2024-09-17T17:29:34.506057+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592515+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:29:33.592515+0000","last_peered":"2024-09-17T17:29:33.592515+0000","last_clean":"2024-09-17T17:29:33.592515+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:29:33.592515+0000","last_undegraded":"2024-09-17T17:29:33.592515+0000","last_fullsized":"2024-09-17T17:29:33.592515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615384+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:29:33.615384+0000","last_peered":"2024-09-17T17:29:33.615384+0000","last_clean":"2024-09-17T17:29:33.615384+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:29:33.615384+0000","last_undegraded":"2024-09-17T17:29:33.615384+0000","last_fullsized":"2024-09-17T17:29:33.615384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.592264+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:29:33.592264+0000","last_peered":"2024-09-17T17:29:33.592264+0000","last_clean":"2024-09-17T17:29:33.592264+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:29:33.592264+0000","last_undegraded":"2024-09-17T17:29:33.592264+0000","last_fullsized":"2024-09-17T17:29:33.592264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:33.615439+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:29:33.615439+0000","last_peered":"2024-09-17T17:29:33.615439+0000","last_clean":"2024-09-17T17:29:33.615439+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:29:33.615439+0000","last_undegraded":"2024-09-17T17:29:33.615439+0000","last_fullsized":"2024-09-17T17:29:33.615439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"25'112","reported_seq":155,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-17T17:29:34.907788+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:29:34.907788+0000","last_peered":"2024-09-17T17:29:34.907788+0000","last_clean":"2024-09-17T17:29:34.907788+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:29:34.907788+0000","last_undegraded":"2024-09-17T17:29:34.907788+0000","last_fullsized":"2024-09-17T17:29:34.907788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,7,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":649012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,7,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,7,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":303104,"data_stored":120103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31484,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":299008,"data_stored":120084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":639466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:29:39.605 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T17:29:39.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T17:29:39.910 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 28 2024-09-17T17:29:39.928 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 2593b77f-a53d-4649-b5a5-0122bd31e2b3 2024-09-17T17:29:39.928 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-17T17:29:15.547925+0000 2024-09-17T17:29:39.928 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-17T17:29:33.585633+0000 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 6 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client jewel 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 25 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.38 2024-09-17T17:29:39.929 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 28 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.02 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: removed_snaps_queue [2~1] 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 16 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/50344815,v1:172.21.15.7:6817/50344815] [v2:172.21.15.7:6818/50344815,v1:172.21.15.7:6819/50344815] exists,up 72ee94e9-57bf-46b3-9fb9-0d05515fe1e1 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6816/2426784045,v1:172.21.15.26:6817/2426784045] [v2:172.21.15.26:6818/2426784045,v1:172.21.15.26:6819/2426784045] exists,up e9764092-a458-4b6d-8bee-658f54f000cc 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/1502510538,v1:172.21.15.27:6801/1502510538] [v2:172.21.15.27:6802/1502510538,v1:172.21.15.27:6803/1502510538] exists,up f1142f7d-9945-4daa-8f6a-6fd2d3090b5f 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6810/2453627254,v1:172.21.15.102:6811/2453627254] [v2:172.21.15.102:6812/2453627254,v1:172.21.15.102:6813/2453627254] exists,up daa4b149-c71a-4cba-a710-89aa101d7d96 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/3507123258,v1:172.21.15.7:6801/3507123258] [v2:172.21.15.7:6802/3507123258,v1:172.21.15.7:6803/3507123258] exists,up 55e8e034-493b-4066-a42b-f4d82294f3f8 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6824/229014119,v1:172.21.15.26:6825/229014119] [v2:172.21.15.26:6826/229014119,v1:172.21.15.26:6827/229014119] exists,up 38a09763-d370-447a-b939-bad8a11c5e01 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 23 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/3536708597,v1:172.21.15.27:6817/3536708597] [v2:172.21.15.27:6818/3536708597,v1:172.21.15.27:6819/3536708597] exists,up 0ae069f9-8831-4bfc-9928-2100b73a839d 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6802/2367087286,v1:172.21.15.102:6803/2367087286] [v2:172.21.15.102:6804/2367087286,v1:172.21.15.102:6805/2367087286] exists,up a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6808/1273991967,v1:172.21.15.7:6809/1273991967] [v2:172.21.15.7:6810/1273991967,v1:172.21.15.7:6811/1273991967] exists,up 3c4daae0-6d93-450f-8600-b29362bd9f77 2024-09-17T17:29:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6808/2309011334,v1:172.21.15.26:6809/2309011334] [v2:172.21.15.26:6810/2309011334,v1:172.21.15.26:6811/2309011334] exists,up 2846b3c6-06d6-414f-b4c9-b16d94007737 2024-09-17T17:29:39.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/2168900921,v1:172.21.15.27:6809/2168900921] [v2:172.21.15.27:6810/2168900921,v1:172.21.15.27:6811/2168900921] exists,up 495d7d7c-c222-4a46-9685-2a2a7cc6fc51 2024-09-17T17:29:39.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6826/3109125505,v1:172.21.15.102:6827/3109125505] [v2:172.21.15.102:6828/3109125505,v1:172.21.15.102:6829/3109125505] exists,up cf76d383-65cd-48c3-a54c-cd83ffbc9898 2024-09-17T17:29:39.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6824/3659919075,v1:172.21.15.7:6825/3659919075] [v2:172.21.15.7:6826/3659919075,v1:172.21.15.7:6827/3659919075] exists,up 54294878-ea1e-4845-a0e6-9be026f521fa 2024-09-17T17:29:39.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6800/1983765258,v1:172.21.15.26:6801/1983765258] [v2:172.21.15.26:6802/1983765258,v1:172.21.15.26:6803/1983765258] exists,up 9acf83a5-75db-4874-8f3f-7421cc55e2fd 2024-09-17T17:29:39.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] [v2:172.21.15.27:6826/1199123829,v1:172.21.15.27:6827/1199123829] exists,up 16c02ebe-a418-4f87-902d-b5dac864f139 2024-09-17T17:29:39.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6818/259571866,v1:172.21.15.102:6819/259571866] [v2:172.21.15.102:6820/259571866,v1:172.21.15.102:6821/259571866] exists,up a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa 2024-09-17T17:29:39.932 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/50344815,v1:172.21.15.7:6817/50344815] [v2:172.21.15.7:6818/50344815,v1:172.21.15.7:6819/50344815] exists,up 72ee94e9-57bf-46b3-9fb9-0d05515fe1e1', 'osd.1 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6816/2426784045,v1:172.21.15.26:6817/2426784045] [v2:172.21.15.26:6818/2426784045,v1:172.21.15.26:6819/2426784045] exists,up e9764092-a458-4b6d-8bee-658f54f000cc', 'osd.2 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/1502510538,v1:172.21.15.27:6801/1502510538] [v2:172.21.15.27:6802/1502510538,v1:172.21.15.27:6803/1502510538] exists,up f1142f7d-9945-4daa-8f6a-6fd2d3090b5f', 'osd.3 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6810/2453627254,v1:172.21.15.102:6811/2453627254] [v2:172.21.15.102:6812/2453627254,v1:172.21.15.102:6813/2453627254] exists,up daa4b149-c71a-4cba-a710-89aa101d7d96', 'osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/3507123258,v1:172.21.15.7:6801/3507123258] [v2:172.21.15.7:6802/3507123258,v1:172.21.15.7:6803/3507123258] exists,up 55e8e034-493b-4066-a42b-f4d82294f3f8', 'osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6824/229014119,v1:172.21.15.26:6825/229014119] [v2:172.21.15.26:6826/229014119,v1:172.21.15.26:6827/229014119] exists,up 38a09763-d370-447a-b939-bad8a11c5e01', 'osd.6 up in weight 1 up_from 23 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/3536708597,v1:172.21.15.27:6817/3536708597] [v2:172.21.15.27:6818/3536708597,v1:172.21.15.27:6819/3536708597] exists,up 0ae069f9-8831-4bfc-9928-2100b73a839d', 'osd.7 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6802/2367087286,v1:172.21.15.102:6803/2367087286] [v2:172.21.15.102:6804/2367087286,v1:172.21.15.102:6805/2367087286] exists,up a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b', 'osd.8 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6808/1273991967,v1:172.21.15.7:6809/1273991967] [v2:172.21.15.7:6810/1273991967,v1:172.21.15.7:6811/1273991967] exists,up 3c4daae0-6d93-450f-8600-b29362bd9f77', 'osd.9 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6808/2309011334,v1:172.21.15.26:6809/2309011334] [v2:172.21.15.26:6810/2309011334,v1:172.21.15.26:6811/2309011334] exists,up 2846b3c6-06d6-414f-b4c9-b16d94007737', 'osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/2168900921,v1:172.21.15.27:6809/2168900921] [v2:172.21.15.27:6810/2168900921,v1:172.21.15.27:6811/2168900921] exists,up 495d7d7c-c222-4a46-9685-2a2a7cc6fc51', 'osd.11 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6826/3109125505,v1:172.21.15.102:6827/3109125505] [v2:172.21.15.102:6828/3109125505,v1:172.21.15.102:6829/3109125505] exists,up cf76d383-65cd-48c3-a54c-cd83ffbc9898', 'osd.12 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6824/3659919075,v1:172.21.15.7:6825/3659919075] [v2:172.21.15.7:6826/3659919075,v1:172.21.15.7:6827/3659919075] exists,up 54294878-ea1e-4845-a0e6-9be026f521fa', 'osd.13 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6800/1983765258,v1:172.21.15.26:6801/1983765258] [v2:172.21.15.26:6802/1983765258,v1:172.21.15.26:6803/1983765258] exists,up 9acf83a5-75db-4874-8f3f-7421cc55e2fd', 'osd.14 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] [v2:172.21.15.27:6826/1199123829,v1:172.21.15.27:6827/1199123829] exists,up 16c02ebe-a418-4f87-902d-b5dac864f139', 'osd.15 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6818/259571866,v1:172.21.15.102:6819/259571866] [v2:172.21.15.102:6820/259571866,v1:172.21.15.102:6821/259571866] exists,up a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa'] 2024-09-17T17:29:39.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T17:29:40.034 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:40.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mon.a", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster": "ceph", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket_mode": "", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_client_required": "cephx, none", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_cluster_required": "cephx", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_debug": "true", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_required": "cephx", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T17:29:40.035 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_supported": "cephx", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio": "true", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_reap_max": "16", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_async_discard_threads": "0", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_block_size": "4096", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio": "false", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_enable_discard": "false", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry": "3", 2024-09-17T17:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash": "0", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring": "false", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_hipri": "false", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_type": "", 2024-09-17T17:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_allocator": "hybrid", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_buffered_io": "true", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T17:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery": "false", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_sync_write": "false", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocator": "hybrid", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_async_db_compaction": "true", 2024-09-17T17:29:40.039 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_create": "true", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_create": "false", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_path": "", 2024-09-17T17:29:40.040 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_size": "0", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_path": "", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_size": "107374182400", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_create": "false", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_path": "", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs": "true", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T17:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune": "true", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T17:29:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size": "0", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_type": "2q", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_clone_cow": "true", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T17:29:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_mode": "none", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_freelist": "false", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T17:29:40.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T17:29:40.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fail_eio": "false", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T17:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T17:29:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size": "0", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T17:29:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T17:29:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_mem": "512", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T17:29:40.050 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_tracing": "false", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_write_v2": "false", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_write_v2_random": "false", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T17:29:40.051 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T17:29:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_version": "2", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_signatures": "false", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_version": "2", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_signatures": "false", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_version": "2", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_sign_messages": "true", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "chdir": "", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_acl_type": "", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_asio_thread_count": "2", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_mid": "0.750000", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_size": "16384", 2024-09-17T17:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_caps_release_delay": "5", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_check_pool_perm": "true", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_force_sync_read": "false", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_getattr_caps": "false", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_features": "", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_remount": "false", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_dirsize_rbytes": "true", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_force_lazyio": "false", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_fs": "", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T17:29:40.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_release_failure": "false", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_inline_size": "4096", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mds_namespace": "", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_metadata": "", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_gid": "-1", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_timeout": "300", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_uid": "-1", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mountpoint": "/", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_notify_timeout": "10", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc": "true", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T17:29:40.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_objects": "1000", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_size": "209715200", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_permissions": "true", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota": "true", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota_df": "true", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_bytes": "0", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_periods": "4", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_min": "131072", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_reconnect_stale": "false", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_shutdown_timeout": "30", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_snapdir": ".snap", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_tick_interval": "1", 2024-09-17T17:29:40.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_trace": "", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_faked_inos": "false", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_random_mds": "false", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog": "false", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_port": "12201", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_monitors": "default=true", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog": "false", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_level": "info", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "-", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network": "", 2024-09-17T17:29:40.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network_interface": "", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_isal": "false", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_level": "5", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zstd_level": "1", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T17:29:40.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T17:29:40.059 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T17:29:40.060 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook": "", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook_timeout": "10", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "daemonize": "false", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "daos_pool": "tank", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_alienstore": "0/5", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok": "1/5", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok_assert_abort": "false", 2024-09-17T17:29:40.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_auth": "20/20", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bdev": "1/3", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluefs": "1/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluestore": "1/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_buffer": "0/1", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_dedup": "0/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephsqlite": "0/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_client": "0/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_compressor": "1/5", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_context": "0/1", 2024-09-17T17:29:40.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crush": "1/1", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crypto": "1/5", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cyanstore": "0/5", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_dpdk": "1/5", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_eventtrace": "1/5", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filer": "0/1", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filestore": "1/3", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_finisher": "1/1", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_fuse": "1/5", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T17:29:40.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_javaclient": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journal": "1/3", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journaler": "0/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_kstore": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_lockdep": "0/1", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mclock": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_balancer": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_locker": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T17:29:40.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_migrator": "1/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_memstore": "1/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgr": "2/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgrc": "1/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mon": "20/20", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_monc": "0/10", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ms": "1/1", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_none": "0/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objclass": "0/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objectcacher": "0/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objecter": "0/1", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_optracker": "0/5", 2024-09-17T17:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_osd": "1/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_paxos": "20/20", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_perfcounter": "1/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_prioritycache": "1/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rados": "0/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd": "0/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_replay": "0/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_refs": "0/0", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_reserver": "1/1", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw": "1/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_access": "1/5", 2024-09-17T17:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_flight": "1/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_notification": "1/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_sync": "1/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rocksdb": "4/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore": "0/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_backref": "0/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cache": "0/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_device": "0/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_epm": "0/5", 2024-09-17T17:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_journal": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_lba": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_odata": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_omap": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_onode": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_t": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_tm": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_striper": "0/1", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_test": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_throttle": "1/1", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_timer": "0/1", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_tp": "0/5", 2024-09-17T17:29:40.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_trace": "1/5", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "device_failure_prediction_mode": "none", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_graylog": "false", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_journald": "false", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_stderr": "true", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_syslog": "false", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event_tracing": "false", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_cert_file": "", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_http_port": "9926", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_key_file": "", 2024-09-17T17:29:40.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_prio_limit": "5", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sort_metrics": "true", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_stats_period": "5", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "fake_statfs_for_testing": "0", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "fatal_signal_handlers": "true", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_purge_ops": "10", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_truncate_ops": "128", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_blackhole": "false", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_snap": "true", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_caller_concurrency": "10", 2024-09-17T17:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_omap_check": "false", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_verify_split": "false", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_dump_file": "", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fadvise": "true", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fail_eio": "true", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_size": "128", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap": "false", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T17:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_inject_stall": "0", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_parallel": "false", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_trailing": "false", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_writeahead": "false", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_kill_at": "0", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T17:29:40.072 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_merge_threshold": "-10", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_odsync_write": "false", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T17:29:40.073 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend_path": "", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_threads": "2", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_punch_hole": "false", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T17:29:40.074 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_ops": "50", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_seek_data_hole": "false", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc": "false", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_splice": "false", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_multiple": "2", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_rand_factor": "20", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_update_to": "1000", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T17:29:40.075 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_xfs_extsize": "false", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_zfs_snap": "false", 2024-09-17T17:29:40.076 INFO:teuthology.orchestra.run.smithi007.stdout: "fio_dir": "/tmp/fio", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "2593b77f-a53d-4649-b5a5-0122bd31e2b3", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_allow_other": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_big_writes": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_debug": "false", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_default_permissions": "false", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_disable_pagecache": "false", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_max_write": "0", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_multithreaded": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_require_active_mds": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_set_user_groups": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_move": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_read": "true", 2024-09-17T17:29:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_write": "true", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_target_name": "ceph", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_file": "", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_inject_failure": "0", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_interval": "5", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "host": "smithi007", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T17:29:40.078 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "inject_early_sigterm": "false", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_agent_port": "6799", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_tracing_enable": "false", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_aio": "true", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_align_min_size": "65536", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_align": "true", 2024-09-17T17:29:40.079 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_size": "4096", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_dio": "true", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_discard": "false", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_force_aio": "false", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_ignore_corruption": "false", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_entries": "100", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_replay_from": "0", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_write_header_frequency": "0", 2024-09-17T17:29:40.080 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_zero_on_create": "false", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prefetch_periods": "10", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prezero_periods": "5", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_write_head_interval": "15", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "keyfile": "", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_backend": "rocksdb", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_bytes": "67108864", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_ops": "512", 2024-09-17T17:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_onode_map_size": "1024", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_transaction": "false", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "librados_thread_count": "2", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep": "false", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep_force_backtrace": "false", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_coarse_timestamps": "true", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_flush_on_exit": "false", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_port": "12201", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_new": "1000", 2024-09-17T17:29:40.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_recent": "10000", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stderr_prefix": "", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": "true", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_graylog": "false", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_journald": "false", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_stderr": "false", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_syslog": "false", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_action_on_write_error": "1", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_alternate_name_max": "8192", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_asio_thread_count": "2", 2024-09-17T17:29:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_export_pin": "true", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_interval": "10", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max": "-1", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max_until": "-1", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_merge_size": "50", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T17:29:40.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_mode": "0", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_bits": "3", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_size": "10000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T17:29:40.085 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_mid": "0.700000", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T17:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_interval": "1", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_auth_pins": "false", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_frag": "false", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_scatterstat": "false", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_subtrees": "false", 2024-09-17T17:29:40.087 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_default_dir_hash": "2", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_defer_session_stale": "true", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_entries": "0", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_prefetch": "true", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T17:29:40.088 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_early_reply": "true", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enable_op_tracker": "true", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enforce_unique_name": "true", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T17:29:40.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_health_dummy": "false", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_join_fs": "", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_journal_format": "1", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_create_at": "0", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_dirfrag_at": "0", 2024-09-17T17:29:40.090 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_export_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_import_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_link_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_openc_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_rename_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T17:29:40.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_events": "-1", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_segments": "128", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_pause": "false", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_segment_size": "0", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_threshold": "128", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T17:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_requests": "100000", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_export_size": "20971520", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_file_recover": "32", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_files": "64", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops": "8192", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_metrics_update_interval": "2", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_per_client": "100", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_numa_node": "-1", 2024-09-17T17:29:40.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_duration": "600", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_size": "20", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_log_threshold": "5", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_grace": "15", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_interval": "5", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_caps": "30000", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T17:29:40.094 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_interval": "1.000000", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_gid": "0", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_uid": "0", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scrub_stats_review_period": "1", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T17:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_shutdown_check": "0", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_skip_ino": "0", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_min_uid": "0", 2024-09-17T17:29:40.096 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_rstat": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_symlink_recovery": "true", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_exports": "0", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_fragments": "0", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_tick_interval": "5.000000", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_valgrind_exit": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_backtrace": "1", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_scatter": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_sessions": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool_debug": "false", 2024-09-17T17:29:40.097 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_set": "false", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_size": "65536", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_bytes": "134217728", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_messages": "512", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_disabled_modules": "", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T17:29:40.098 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_creating": "1024", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_messages": "128", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_messages": "128", 2024-09-17T17:29:40.099 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_duration": "600", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_size": "20", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_log_threshold": "5", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_messages": "8192", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_pool": "true", 2024-09-17T17:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_standby_modules": "true", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_period": "5", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_threshold": "5", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_tick_period": "2", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_delete": "true", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_bytes": "104857600", 2024-09-17T17:29:40.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_target_rank": "-1", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T17:29:40.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_level": "debug", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_start": "false", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_trim": "true", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T17:29:40.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cpu_threads": "4", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_crit": "5", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_warn": "5", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_json": "false", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T17:29:40.105 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_extra_checks": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_uptime_grace": "60", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_default_strategy": "1", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_timeout": "5.000000", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T17:29:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_enable_op_tracker": "true", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fake_pool_delete": "false", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_force_quorum_join": "false", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_log_update_period": "5", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_max_detail": "50", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog": "true", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T17:29:40.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host": "172.21.15.7,172.21.15.26,172.21.15.27", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host_override": "", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_initial_members": "", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease": "5.000000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_full_interval": "50", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max": "10000", 2024-09-17T17:29:40.108 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max_summary": "50", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_epochs": "500", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_osd": "10000", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_autotune": "true", 2024-09-17T17:29:40.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_target": "2147483648", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_digest_period": "5", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_complaint_time": "30", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_duration": "600", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_size": "20", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T17:29:40.110 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_log_threshold": "5", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size": "500", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T17:29:40.111 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T17:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_report_timeout": "900", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T17:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_osds": "4", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T17:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_interval": "300", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_max_keys": "100", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_timeout": "300", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_session_timeout": "300", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_smart_report_timeout": "5", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_size": "4", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T17:29:40.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_debug": "false", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_tick_interval": "5", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T17:29:40.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_older_version": "true", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T17:29:40.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": "", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_tracing_enabled": "false", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_op_threads": "3", 2024-09-17T17:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_cm": "false", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_device_name": "", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_sl": "3", 2024-09-17T17:29:40.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_type": "ib", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_reap_threshold": "5", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_before_connect": "false", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv4": "true", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv6": "false", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr1": "true", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr2": "true", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_max": "7568", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_min": "6800", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_count": "3", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_delay": "5", 2024-09-17T17:29:40.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_client": "false", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mds": "false", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mgr": "false", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mon": "false", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_osd": "false", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_mode": "crc secure", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_type": "", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_compress_secure": "false", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_data": "true", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_header": "true", 2024-09-17T17:29:40.121 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bug": "true", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_old_message": "true", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hugepages": "", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T17:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_lro": "true", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_pmd": "", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_port_id": "0", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_on_send": "false", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_probability": "0.005000", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_type": "osd", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_internal_delays": "0.002000", 2024-09-17T17:29:40.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_network_congestion": "0", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_socket_failures": "2500", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_accept_failures": "4", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_backoff": "15.000000", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_mode": "none", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_min_cost": "65536", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_public_type": "", 2024-09-17T17:29:40.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_service_mode": "crc secure", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_nodelay": "true", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_type": "async+posix", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "no_config_file": "false", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T17:29:40.125 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_ops": "1024", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_timeout": "10.000000", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_blackhole": "false", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "openssl_engine_opts": "", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_ops": "4", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T17:29:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_slop": "0.020000", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_asio_thread_count": "2", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_async_recovery_min_cost": "1", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_max": "512", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_min": "64", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_peering": "false", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_beacon_report_interval": "300", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_duration": "30", 2024-09-17T17:29:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blkin_trace_all": "false", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T17:29:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_update_on_start": "true", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_cap": "256", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_op_priority": "63", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_watch_timeout": "30", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_command_max_records": "256", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_compact_on_start": "false", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_on_start": "true", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T17:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_op_order": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T17:29:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_shutdown": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_trim_objects": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T17:29:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T17:29:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T17:29:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T17:29:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T17:29:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_notify_timeout": "30", 2024-09-17T17:29:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ec_partial_reads": "true", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_enable_op_tracker": "true", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_info": "true", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown": "true", 2024-09-17T17:29:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_function_tracing": "false", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_grace": "20", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_interval": "6", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_stale": "600", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T17:29:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_size": "5120", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_kill_backfill_at": "0", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_cache_size": "50", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_dedup": "true", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max": "40", 2024-09-17T17:29:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_name_len": "100", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_size": "0", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_backfills": "1", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_count": "5", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_period": "600", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_name_len": "2048", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_size": "134217728", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T17:29:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pgls": "1024", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_cost": "8388608", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_objects": "10", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_scrubs": "3", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_size": "90", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T17:29:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_profile": "balanced", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T17:29:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_base": "805306368", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target": "4294967296", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_autotune": "false", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_recovery_priority": "0", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T17:29:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_interval": "5", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_cache_shards": "32", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_node": "-1", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objecter_finishers": "1", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore": "bluestore", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_fuse": "false", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-17T17:29:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_tracing": "false", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_duration": "600", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_size": "20", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_log_threshold": "5", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards": "0", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T17:29:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_timeout": "15", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_open_classes_on_start": "true", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_os_flags": "0", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_peering_op_priority": "255", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_priority": "5", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T17:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crimson": "false", 2024-09-17T17:29:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flags": "0", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_min_size": "0", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T17:29:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_size": "2", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_type": "replicated", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_cost": "20971520", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active": "0", 2024-09-17T17:29:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_priority": "3", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_priority": "5", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_repair_during_recovery": "false", 2024-09-17T17:29:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_max": "15", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_cost": "52428800", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T17:29:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_hour": "0", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_priority": "5", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_delay": "30", 2024-09-17T17:29:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_trimming": "10", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_data_digest": "false", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_smart_report_timeout": "5", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_priority": "5", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T17:29:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_transaction_size": "30", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T17:29:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tracing": "false", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_use_stale_snap": "false", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_kill_at": "0", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_max_join_drift": "10", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min": "500", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min_wait": "0.050000", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max": "500", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T17:29:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_min": "250", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_stash_full_interval": "25", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_max": "500", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_min": "250", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "perf": "true", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "-", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrv": "", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_bind_addr": "-", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network": "", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network_interface": "", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T17:29:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_enabled": "false", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_mon_op_timeout": "0", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_osd_op_timeout": "0", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_tracing": "false", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_atime_update_interval": "60", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache": "true", 2024-09-17T17:29:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_policy": "writearound", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_size": "33554432", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_compression_hint": "none", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_clone_format": "auto", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_data_pool": "", 2024-09-17T17:29:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_features": "61", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_format": "2", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_map_options": "", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_order": "22", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_pool": "rbd", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_count": "0", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler": "simple", 2024-09-17T17:29:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_order": "24", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_pool": "", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_splay_width": "4", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T17:29:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T17:29:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T17:29:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_threads": "1", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_plugins": "", 2024-09-17T17:29:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T17:29:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T17:29:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_tracing": "false", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_names": "true", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_pool": "true", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_base_url": "", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_log_level": "", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_admin_entry": "admin", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_asio_assert_yielding": "false", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_backend_store": "rados", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_barbican_url": "", 2024-09-17T17:29:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_beast_enable_async": "true", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_enabled": "true", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_config_store": "rados", 2024-09-17T17:29:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_content_length_compat": "false", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cross_domain_policy": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T17:29:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T17:29:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T17:29:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-17T17:29:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_window": "30", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T17:29:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_disable_s3select": "false", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T17:29:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_name": "", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_s3website_name": "", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T17:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_mdsearch": "true", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_ops_log": "false", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_static_website": "false", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_usage_log": "false", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T17:29:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_expose_bucket": "false", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_extended_http_attrs": "", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_filter": "none", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_objs": "32", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T17:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_init_timeout": "300", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T17:29:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_project": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_user": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T17:29:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_url": "", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_objs": "32", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_worker": "3", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T17:29:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_uri": "", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_http_headers": "", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T17:29:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_size": "0", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_listing_results": "5000", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_notify_retries": "10", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T17:29:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T17:29:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_control_oids": "8", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_numa_node": "-1", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T17:29:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_tracing": "false", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_token": "", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_url": "", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_rados": "false", 2024-09-17T17:29:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T17:29:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_continue": "true", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_tracing": "false", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm": "", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_id": "", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region": "", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T17:29:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_request_uri": "", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_resolve_cname": "false", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T17:29:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_run_sync_thread": "true", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_script_uri": "", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_service_provider_name": "", 2024-09-17T17:29:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_id": "", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_secret": "", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_entry": "sts", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_key": "sts", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_url": "", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_custom_header": "", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_need_stats": "true", 2024-09-17T17:29:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_tenant_name": "", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url": "", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_lease_period": "120", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T17:29:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_thread_pool_size": "512", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_comment": "", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_createby": "", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_encoding": "", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_flag": "false", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T17:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_origin": "", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_tracker": "", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_shards": "32", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_use_opa_authz": "false", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache": "false", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T17:29:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_unique_email": "true", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_verify_ssl": "true", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone": "", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_id": "", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup": "", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_id": "", 2024-09-17T17:29:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_block_size": "4096", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T17:29:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_partition_filters": "false", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_perf": "false", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "run_dir": "/var/run/ceph", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_block_create": "true", 2024-09-17T17:29:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_device_size": "53687091200", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_full_integrity_check": "false", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_main_device_type": "SSD", 2024-09-17T17:29:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_segment_size": "67108864", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "set_keepcaps": "false", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "setgroup": "", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser": "", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser_match_path": "", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "thp": "false", 2024-09-17T17:29:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_default_timeout": "60", 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "throttler_perf_counter": "true", 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tmp_dir": "/tmp", 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uadk_compressor_enabled": "false", 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-17T17:29:40.194 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:40.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T17:29:40.323 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-17T17:29:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T17:29:40.539 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-17T17:29:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T17:29:40.554 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-17T17:29:40.554 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T17:29:40.565 DEBUG:teuthology.orchestra.run.smithi027:> type ceph-objectstore-tool 2024-09-17T17:29:40.585 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T17:29:40.585 DEBUG:teuthology.orchestra.run.smithi007:> type ceph-objectstore-tool 2024-09-17T17:29:40.601 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T17:29:40.601 DEBUG:teuthology.orchestra.run.smithi102:> type ceph-objectstore-tool 2024-09-17T17:29:40.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T17:29:40.620 DEBUG:teuthology.orchestra.run.smithi026:> type ceph-objectstore-tool 2024-09-17T17:29:40.641 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T17:29:40.641 INFO:teuthology.run_tasks:Running task rados... 2024-09-17T17:29:40.654 INFO:tasks.rados:Beginning rados... 2024-09-17T17:29:40.654 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-17T17:29:40.655 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath10-testing-2024-09-17-1348'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath10-testing-2024-09-17-1348', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}} 2024-09-17T17:29:40.655 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-17T17:29:40.655 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-17T17:29:40.701 INFO:tasks.rados:joining rados 2024-09-17T17:29:40.701 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T17:29:40.701 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:29:40.701 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:29:40.702 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T17:29:40.702 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T17:29:40.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:29:40.703 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T17:29:40.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:29:40.703 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T17:29:40.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:29:40.704 INFO:tasks.rados:clients are ['client.0'] 2024-09-17T17:29:40.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-09-17T17:29:40.706 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:40.701+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:40.805 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:40.802+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:40.821 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:29:40.822 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:40.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:29:40.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:40.850 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:40.873 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:40.873 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:40.896 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:40.896 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:40.906 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:40.904+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:40.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:40.955 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:40.956 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:40.958 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:40.958 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:40.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:29:40.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:41.006 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:41.004+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:41.093 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.094 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:29:41.107 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:41.105+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:41.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:41.154 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:41.176 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:41.177 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:41.207 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:41.204+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:41.215 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.215 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:29:41.309 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:41.307+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:41.324 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:29:41.409+0000 7fe717fff640 0 -- 172.21.15.7:0/1707674447 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fe720221820 msgr2=0x7fe720223c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:29:41.411 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:41.409+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:41.433 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:41.433 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:29:41.454 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:41.510 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:41.508+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:41.512 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.512 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:29:41.611 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:41.609+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:41.618 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:29:41.618 INFO:teuthology.orchestra.run.smithi007.stderr:load: shec 2024-09-17T17:29:41.636 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.636 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.637 INFO:tasks.rados:starting run 0 out of 1 2024-09-17T17:29:41.638 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-17T17:29:41.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-09-17T17:29:41.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:29:41.729 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:41.727+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:29:41.845 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:41.843+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:41.913 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:41.911+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:41.953 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:41.953 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:41.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:29:42.014 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:42.012+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:29:42.122 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:42.120+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:42.184 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.185 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:29:42.235 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:42.233+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:42.319 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:42.317+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:42.341 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.341 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:29:42.419 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:42.417+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:42.470 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.470 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.479 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:29:42.521 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:42.519+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:42.587 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.588 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:29:42.621 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:42.619+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:42.703 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.703 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.712 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:29:42.722 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:42.720+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:42.819 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.820 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.823 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:42.821+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:42.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:29:42.924 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:42.922+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:42.975 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:42.976 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:42.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:29:43.028 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:43.026+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:43.091 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.091 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:29:43.140 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:43.138+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:43.227 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:43.224+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:43.247 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.247 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:29:43.327 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:43.326+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:43.363 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.363 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:29:43.429 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:43.427+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:43.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:29:43.478 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.478 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:29:43.530 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:43.528+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:43.594 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.594 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:29:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:43.615 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:43.630 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'unique_pool_0' created 2024-09-17T17:29:43.631 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:43.628+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:43.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:29:43.640 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:43.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-17T17:29:43.668 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:43.669 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:43.691 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:43.691 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:43.712 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:43.716 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.717 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:29:43.731 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:43.729+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:43.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:43.759 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:43.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:43.803 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:43.804 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:43.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:43.825 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:43.832 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:43.830+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:43.833 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:43.834 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:43.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:29:43.846 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:43.867 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:43.867 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:43.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:43.910 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:43.910 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:43.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:29:43.933 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:43.931+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:43.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:43.962 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:43.962 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:43.962 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:29:43.962 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:43.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:29:44.034 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:44.031+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:44.118 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:29:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:29:44.118 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:44.127 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:29:44.134 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:44.132+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:44.233 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:44.233 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:29:44.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:29:44.233 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [] 2024-09-17T17:29:44.234 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:44.235 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:44.233+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:44.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:29:44.336 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:44.333+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:44.348 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:44.348 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:44.348 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:29:44.349 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:44.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:29:44.437 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:44.434+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:44.463 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:44.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:29:44.464 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:29:44.464 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:44.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:29:44.537 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:44.535+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:44.580 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.997089627999999, 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.27275196200000001, 2024-09-17T17:29:44.581 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1 2024-09-17T17:29:44.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580026+0000", 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.359e-06 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580034+0000", 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580048+0000", 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3974e-05 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580058+0000", 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0064999999999999e-05 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580111+0000", 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3248000000000002e-05 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580119+0000", 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1270000000000003e-06 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847220+0000", 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26710079599999997 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847357+0000", 2024-09-17T17:29:44.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013739500000000001 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847505+0000", 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147188 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.850265+0000", 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002760396 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852701+0000", 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024356059999999999 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852744+0000", 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3095e-05 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852777+0000", 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2784000000000003e-05 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.986961+0000", 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.590152797, 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067482250000000001, 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 41 2024-09-17T17:29:44.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.592 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.986961+0000", 2024-09-17T17:29:44.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.986961+0000", 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.986962+0000", 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5099999999999999e-07 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987036+0000", 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4197999999999997e-05 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987044+0000", 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3719999999999999e-06 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987048+0000", 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5680000000000001e-06 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987890+0000", 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084162500000000003 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987952+0000", 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1723999999999994e-05 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.988048+0000", 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6335999999999994e-05 2024-09-17T17:29:44.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.989712+0000", 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016640489999999999 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993629+0000", 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039173319999999999 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993665+0000", 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5840000000000002e-05 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993710+0000", 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4530000000000002e-05 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987266+0000", 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.589848570999999, 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067434729999999998, 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 44 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987266+0000", 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987266+0000", 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987266+0000", 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5799999999999998e-07 2024-09-17T17:29:44.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987346+0000", 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0057999999999998e-05 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987353+0000", 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8310000000000003e-06 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987358+0000", 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3590000000000001e-06 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.989268+0000", 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001910772 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.989321+0000", 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2357e-05 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.989407+0000", 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6348999999999996e-05 2024-09-17T17:29:44.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990777+0000", 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001370111 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993884+0000", 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031063190000000002 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993963+0000", 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9062999999999998e-05 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994009+0000", 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6495999999999997e-05 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.606 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.589718649, 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010915796, 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 46 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5400000000000002e-07 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987400+0000", 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1450000000000001e-06 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987405+0000", 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2739999999999998e-06 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987409+0000", 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1370000000000004e-06 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990238+0000", 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028288110000000001 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990293+0000", 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4657999999999997e-05 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990394+0000", 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010156 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991880+0000", 2024-09-17T17:29:44.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014857169999999999 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998207+0000", 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0063267899999999997 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998262+0000", 2024-09-17T17:29:44.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998311+0000", 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9777999999999997e-05 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.614 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.589685534000001, 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010963481000000001, 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 47 2024-09-17T17:29:44.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5400000000000001e-07 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987478+0000", 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9118e-05 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987490+0000", 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1425e-05 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987502+0000", 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2354999999999999e-05 2024-09-17T17:29:44.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990908+0000", 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003406002 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990958+0000", 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9597e-05 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991063+0000", 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010533 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993063+0000", 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019997909999999999 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998320+0000", 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052566959999999999 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998349+0000", 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9200000000000002e-05 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998392+0000", 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3412999999999999e-05 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.622 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.58960134, 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010965536999999999, 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 48 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.623 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5899999999999996e-07 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987518+0000", 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0429999999999998e-06 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987523+0000", 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3499999999999999e-06 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987526+0000", 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5159999999999999e-06 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991339+0000", 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038128670000000002 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991396+0000", 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7219000000000002e-05 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991498+0000", 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010142 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993114+0000", 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016166900000000001 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998400+0000", 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052850600000000003 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998438+0000", 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8272000000000002e-05 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998478+0000", 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0540999999999997e-05 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:44.630 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.58958299, 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011028196000000001, 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 49 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987534+0000", 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4399999999999999e-06 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987538+0000", 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9369999999999997e-06 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987541+0000", 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0350000000000002e-06 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991920+0000", 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043794000000000003 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991972+0000", 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1412000000000002e-05 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992057+0000", 2024-09-17T17:29:44.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4917e-05 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993157+0000", 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011003810000000001 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998485+0000", 2024-09-17T17:29:44.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053282989999999999 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998517+0000", 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2019e-05 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998559+0000", 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2076000000000002e-05 2024-09-17T17:29:44.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.589561501, 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011080325, 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 50 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:44.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-07 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987605+0000", 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2114000000000003e-05 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987631+0000", 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5766999999999999e-05 2024-09-17T17:29:44.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987646+0000", 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515999999999998e-05 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992371+0000", 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047247360000000002 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992433+0000", 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1451000000000005e-05 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992522+0000", 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9621000000000001e-05 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994510+0000", 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001988227 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998564+0000", 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040533710000000001 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998595+0000", 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0843000000000003e-05 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998633+0000", 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8321000000000001e-05 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.589449476, 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011043702000000001, 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 51 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:44.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987669+0000", 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7189999999999999e-06 2024-09-17T17:29:44.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987674+0000", 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1780000000000002e-06 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987678+0000", 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5999999999999998e-06 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993192+0000", 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055137620000000002 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993262+0000", 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993349+0000", 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7362999999999997e-05 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994549+0000", 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011994110000000001 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998640+0000", 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040912409999999998 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998669+0000", 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9209000000000001e-05 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998708+0000", 2024-09-17T17:29:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9419000000000003e-05 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.589429386000001, 2024-09-17T17:29:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011098023, 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 52 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2000000000000001e-07 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987688+0000", 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4729999999999999e-06 2024-09-17T17:29:44.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987693+0000", 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5010000000000004e-06 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987698+0000", 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1499999999999998e-06 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994130+0000", 2024-09-17T17:29:44.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064314890000000003 2024-09-17T17:29:44.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994185+0000", 2024-09-17T17:29:44.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5769000000000001e-05 2024-09-17T17:29:44.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994272+0000", 2024-09-17T17:29:44.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6638000000000006e-05 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994990+0000", 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000717463 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998715+0000", 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003725 2024-09-17T17:29:44.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998743+0000", 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8830999999999999e-05 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998783+0000", 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9388999999999998e-05 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:58 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [cmpxattr striper.excl (26) op 1 mode 1 in=12b,setxattr striper.excl in=12b,call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.005578+0000", 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.571535806, 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0082001399999999999, 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.664 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 58 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005578+0000", 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005578+0000", 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005579+0000", 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.051e-06 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005584+0000", 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9350000000000002e-06 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005595+0000", 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1060000000000001e-05 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005601+0000", 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3140000000000003e-06 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005647+0000", 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6548999999999998e-05 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005704+0000", 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6317000000000003e-05 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005837+0000", 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013316399999999999 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.006293+0000", 2024-09-17T17:29:44.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00045653000000000002 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.013713+0000", 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074199009999999996 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.013750+0000", 2024-09-17T17:29:44.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6857000000000003e-05 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.013779+0000", 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8462e-05 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.296826+0000", 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.280288369000001, 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076706259999999998, 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 69 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296826+0000", 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296826+0000", 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296827+0000", 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.71e-07 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296949+0000", 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012259700000000001 2024-09-17T17:29:44.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296964+0000", 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4651000000000001e-05 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296973+0000", 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8550000000000008e-06 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.299602+0000", 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026291550000000002 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.299693+0000", 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1521999999999996e-05 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.299855+0000", 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000161817 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301550+0000", 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016943609999999999 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304409+0000", 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028592309999999998 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304449+0000", 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0623000000000001e-05 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304496+0000", 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6943000000000001e-05 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.296990+0000", 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.280123864, 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075928840000000003, 2024-09-17T17:29:44.679 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 70 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296990+0000", 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296990+0000", 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296991+0000", 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296997+0000", 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5380000000000002e-06 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297007+0000", 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7629999999999995e-06 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297013+0000", 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1360000000000002e-06 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.300309+0000", 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0032966720000000001 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.300363+0000", 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3461000000000002e-05 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.300454+0000", 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1452999999999993e-05 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301599+0000", 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011443720000000001 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304508+0000", 2024-09-17T17:29:44.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002909029 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304539+0000", 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1778e-05 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304583+0000", 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3788999999999997e-05 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.686 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.280089471, 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076388319999999999, 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 71 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:44.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5700000000000002e-07 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297030+0000", 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5430000000000004e-06 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297038+0000", 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4740000000000005e-06 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297045+0000", 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1630000000000001e-06 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301114+0000", 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040693709999999996 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301166+0000", 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1748e-05 2024-09-17T17:29:44.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301255+0000", 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9326999999999994e-05 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304399+0000", 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031437930000000002 2024-09-17T17:29:44.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304591+0000", 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019223600000000001 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304625+0000", 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3560999999999999e-05 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304664+0000", 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8958999999999997e-05 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297056+0000", 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.280057867, 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081331440000000001, 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.695 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 72 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297056+0000", 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297056+0000", 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297057+0000", 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7399999999999998e-07 2024-09-17T17:29:44.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297172+0000", 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011553900000000001 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297209+0000", 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6609000000000001e-05 2024-09-17T17:29:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297219+0000", 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7480000000000005e-06 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301634+0000", 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044148679999999997 2024-09-17T17:29:44.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301678+0000", 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4739e-05 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301759+0000", 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1100000000000006e-05 2024-09-17T17:29:44.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304714+0000", 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029548719999999999 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305111+0000", 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000396876 2024-09-17T17:29:44.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305145+0000", 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3424999999999999e-05 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305189+0000", 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4894000000000001e-05 2024-09-17T17:29:44.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297253+0000", 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.279860802, 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087998280000000009, 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 73 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297253+0000", 2024-09-17T17:29:44.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297253+0000", 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297254+0000", 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5600000000000005e-07 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297258+0000", 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2080000000000002e-06 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297266+0000", 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5190000000000003e-06 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297272+0000", 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7910000000000001e-06 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304008+0000", 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067363470000000002 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304061+0000", 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3219e-05 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304142+0000", 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0717999999999995e-05 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305957+0000", 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018147510000000001 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305969+0000", 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.252e-05 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306004+0000", 2024-09-17T17:29:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4813e-05 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306053+0000", 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9286000000000002e-05 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.279835263000001, 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093072850000000002, 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 74 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:44.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4200000000000001e-07 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297284+0000", 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8999999999999997e-06 2024-09-17T17:29:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297289+0000", 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4580000000000003e-06 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297293+0000", 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7339999999999999e-06 2024-09-17T17:29:44.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304762+0000", 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074683170000000004 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304811+0000", 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8992999999999997e-05 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304894+0000", 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2805000000000004e-05 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306104+0000", 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012108119999999999 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306502+0000", 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039784999999999999 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306541+0000", 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8779000000000003e-05 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306586+0000", 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5194999999999997e-05 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.279813462, 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093839609999999997, 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 75 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:44.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-17T17:29:44.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297372+0000", 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1192000000000005e-05 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297381+0000", 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000003e-06 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297386+0000", 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7999999999999998e-06 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305232+0000", 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078461910000000006 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305284+0000", 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2522999999999998e-05 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305373+0000", 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8889999999999998e-05 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306598+0000", 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012251969999999999 2024-09-17T17:29:44.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306630+0000", 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1718999999999999e-05 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306662+0000", 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1458999999999998e-05 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306685+0000", 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.304e-05 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.279714891999999, 2024-09-17T17:29:44.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011397086000000001, 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 76 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297400+0000", 2024-09-17T17:29:44.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7100000000000002e-07 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297403+0000", 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5920000000000001e-06 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297410+0000", 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1489999999999997e-06 2024-09-17T17:29:44.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297417+0000", 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-06 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306131+0000", 2024-09-17T17:29:44.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087138279999999999 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306185+0000", 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3783999999999999e-05 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306278+0000", 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3587999999999999e-05 2024-09-17T17:29:44.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306994+0000", 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071571099999999995 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308729+0000", 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001735489 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308769+0000", 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9270000000000002e-05 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308796+0000", 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7603999999999999e-05 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.732 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:124 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.03765334, 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011177351, 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:44.733 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 124 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539467+0000", 2024-09-17T17:29:44.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2089999999999996e-06 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539473+0000", 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4690000000000002e-06 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539479+0000", 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-17T17:29:44.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545059+0000", 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055806400000000004 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545104+0000", 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5040999999999998e-05 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545172+0000", 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7249999999999995e-05 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545874+0000", 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070271799999999996 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550512+0000", 2024-09-17T17:29:44.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046379530000000002 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550597+0000", 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025999999999995e-05 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550638+0000", 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0771e-05 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:44.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:44.740 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:44.741 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:29:44.741 INFO:teuthology.orchestra.run.smithi007.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-17T17:29:44.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:29:44.743 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:44.740+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:44.843 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:44.841+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:44.893 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:44.893 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:44.893 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:29:44.894 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:44.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:29:44.943 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:44.942+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:45.009 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:45.009 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:45.009 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:29:45.009 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:29:45.009 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:45.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:29:45.062 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:45.061+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:45.124 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:45.124 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:29:45.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:29:45.125 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [] 2024-09-17T17:29:45.125 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:45.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:29:45.145 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:45.143+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:45.246 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:45.244+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:45.281 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:45.281 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:29:45.281 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:45.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:29:45.347 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:45.345+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:45.396 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:29:45.396 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:29:45.396 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:45.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:29:45.448 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:45.445+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:45.548 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:45.545+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:45.551 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:29:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:29:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:29:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.551 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4311.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:31.586327+0000", 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.963853285999999, 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003699506, 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4311", 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.7:0/2554837321", 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1 2024-09-17T17:29:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586327+0000", 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586327+0000", 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:45.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586336+0000", 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4509999999999992e-06 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586343+0000", 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7349999999999996e-06 2024-09-17T17:29:45.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586353+0000", 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7799999999999995e-06 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586362+0000", 2024-09-17T17:29:45.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4040000000000001e-06 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586412+0000", 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9246000000000003e-05 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.586418+0000", 2024-09-17T17:29:45.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0120000000000002e-06 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.587512+0000", 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001094072 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:29:45.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.587517+0000", 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4360000000000001e-06 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.588866+0000", 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001348868 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:45.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589000+0000", 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013413100000000001 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 15", 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589160+0000", 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000159856 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589699+0000", 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053856800000000001 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589974+0000", 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027523100000000002 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.590004+0000", 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0577999999999998e-05 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.590027+0000", 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2138e-05 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:45.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop_reply(client.4311.0:1 2.3 e26/25 ondisk, result = 0)", 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:31.589886+0000", 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.960293962, 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014599700000000001, 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:45.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589886+0000", 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589886+0000", 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:45.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589887+0000", 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3900000000000004e-07 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589892+0000", 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6129999999999997e-06 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589921+0000", 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9149000000000002e-05 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589931+0000", 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9450000000000005e-06 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589966+0000", 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5533e-05 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.589971+0000", 2024-09-17T17:29:45.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6439999999999999e-06 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:31.590032+0000", 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1373999999999999e-05 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:45.567 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:45.568 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:45.569 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:29:45.649 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:45.648+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:45.682 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:29:45.682 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:29:45.682 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:29:45.682 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:29:45.690 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:29:45.702 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:29:45.702 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:29:45.703 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-17T17:29:45.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T17:29:45.745 DEBUG:teuthology.orchestra.run.smithi102:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-17T17:29:45.750 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:45.748+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight read -> 100 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight delete -> 50 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight snap_create -> 50 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight snap_remove -> 50 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight rollback -> 50 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight setattr -> 25 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight rmattr -> 25 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight copy_from -> 50 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:adding op weight append -> 100 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:ceph version 19.3.0-4975-g1889a165 (1889a165b5128787ec693450a3d1c5df5d6dc203) squid (dev) 2024-09-17T17:29:45.773 INFO:tasks.rados.rados.0.smithi102.stdout:Configuration: 2024-09-17T17:29:45.774 INFO:tasks.rados.rados.0.smithi102.stdout: Number of operations: 400 2024-09-17T17:29:45.774 INFO:tasks.rados.rados.0.smithi102.stdout: Number of objects: 50 2024-09-17T17:29:45.774 INFO:tasks.rados.rados.0.smithi102.stdout: Max in flight operations: 16 2024-09-17T17:29:45.774 INFO:tasks.rados.rados.0.smithi102.stdout: Object size (in bytes): 4000000 2024-09-17T17:29:45.774 INFO:tasks.rados.rados.0.smithi102.stdout: Write stride min: 400000 2024-09-17T17:29:45.774 INFO:tasks.rados.rados.0.smithi102.stdout: Write stride max: 800000 2024-09-17T17:29:45.796 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:29:45.796 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:29:45.796 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:29:45.796 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:29:45.796 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:29:45.805 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:29:45.889 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:45.886+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4311.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:31.597165+0000", 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 14.350407855, 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00030389099999999998, 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4311", 2024-09-17T17:29:45.950 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.7:0/2554837321", 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 2 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597165+0000", 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597165+0000", 2024-09-17T17:29:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597180+0000", 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5213e-05 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597193+0000", 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3173000000000001e-05 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.952 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597215+0000", 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1642999999999999e-05 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597233+0000", 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7626e-05 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.953 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597357+0000", 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012454799999999999 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597469+0000", 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00011168800000000001 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.954 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4311.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:31.597756+0000", 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 14.349816374, 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000157054, 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4311", 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.7:0/2554837321", 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 3 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.955 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597756+0000", 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.956 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597756+0000", 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597757+0000", 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:29:45.957 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597760+0000", 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8030000000000001e-06 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597768+0000", 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9070000000000007e-06 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597772+0000", 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3730000000000003e-06 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.958 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597843+0000", 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0390999999999997e-05 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:31.597913+0000", 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0871999999999995e-05 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.959 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4311.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:32.593579+0000", 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 13.353993765, 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043395899999999999, 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4311", 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.7:0/2554837321", 2024-09-17T17:29:45.960 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 5 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593579+0000", 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593579+0000", 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593581+0000", 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0080000000000001e-06 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593590+0000", 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4849999999999992e-06 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593606+0000", 2024-09-17T17:29:45.962 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6779e-05 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593620+0000", 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3919e-05 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593700+0000", 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9684000000000005e-05 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.593707+0000", 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.9249999999999998e-06 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.595884+0000", 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0021769659999999998 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.596051+0000", 2024-09-17T17:29:45.964 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000166918 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for subops from 9", 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.596235+0000", 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00018427800000000001 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op_commit", 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597246+0000", 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00101088 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.965 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597807+0000", 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00056143499999999999 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597874+0000", 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6531000000000004e-05 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597919+0000", 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4781999999999999e-05 2024-09-17T17:29:45.966 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_repop_reply(client.4311.0:5 2.2 e26/25 ondisk, result = 0)", 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:32.597675+0000", 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 13.349897544999999, 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00025308999999999998, 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:29:45.967 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597675+0000", 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597675+0000", 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597676+0000", 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.085e-06 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597682+0000", 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9089999999999997e-06 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597724+0000", 2024-09-17T17:29:45.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2237000000000002e-05 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597733+0000", 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5329999999999994e-06 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597787+0000", 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3718000000000003e-05 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.970 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597798+0000", 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1727e-05 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:32.597928+0000", 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012988100000000001 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.971 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4311.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:33.592580+0000", 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.354992994, 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0014073709999999999, 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4311", 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.7:0/2554837321", 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 7 2024-09-17T17:29:45.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592580+0000", 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592580+0000", 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592581+0000", 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4389999999999999e-06 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592589+0000", 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9189999999999999e-06 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592603+0000", 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3755e-05 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592613+0000", 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0210000000000001e-05 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592666+0000", 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2621000000000003e-05 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592747+0000", 2024-09-17T17:29:45.975 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1363e-05 2024-09-17T17:29:45.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for subops from 9", 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.592864+0000", 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00011670199999999999 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op_commit", 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593497+0000", 2024-09-17T17:29:45.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00063330899999999998 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593923+0000", 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042634599999999997 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593957+0000", 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3815000000000001e-05 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593987+0000", 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9892000000000001e-05 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_repop_reply(client.4311.0:7 2.2 e28/25 ondisk, result = 0)", 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:33.593822+0000", 2024-09-17T17:29:45.979 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.353750571000001, 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016981900000000001, 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593822+0000", 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.980 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593822+0000", 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593823+0000", 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4e-07 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:29:45.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593837+0000", 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4273e-05 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593868+0000", 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0855999999999997e-05 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593875+0000", 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6479999999999997e-06 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593913+0000", 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8209e-05 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593919+0000", 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6550000000000001e-06 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:33.593992+0000", 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3238000000000006e-05 2024-09-17T17:29:45.983 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.984 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.984 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.984 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:29:45.984 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:29:45.984 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:29:45.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:29:45.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:29:45.987 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:45.983+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:46.033 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T17:29:46.031+0000 7f873ad1c640 -1 mon.a@0(leader).osd e34 definitely_dead 0 2024-09-17T17:29:46.089 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:46.087+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:46.119 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:46.130 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:29:46.130 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:29:46.131 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:29:46.131 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:29:46.139 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:29:46.142 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:46.142 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:46.163 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:46.164 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:46.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:46.190 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:46.188+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:46.205 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:46.227 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:46.248 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:46.270 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:46.283 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:46.283 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:46.290 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:46.290 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:46.291 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:46.289+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:46.291 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:29:46.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:46.332 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:46.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:46.355 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:46.392 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:46.390+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:46.395 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:46.395 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:29:46.401+0000 7fe52dd9b640 0 -- 172.21.15.7:0/2642089605 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fe5302211c0 msgr2=0x7fe5302235b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:29:46.404 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:29:46.445 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:46.493 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:46.492+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:46.507 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:46.508 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:46.516 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:29:46.594 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:46.592+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:46.619 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:46.619 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:46.628 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:29:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.2. 2024-09-17T17:29:46.652 INFO:tasks.rados.rados.0.smithi102.stdout:1: write initial oid 1 2024-09-17T17:29:46.652 INFO:tasks.rados.rados.0.smithi102.stdout:1: seq_num 0 ranges {0=32768} 2024-09-17T17:29:46.652 INFO:tasks.rados.rados.0.smithi102.stdout:1: writing smithi10238019-1 from 0 to 32768 tid 1 2024-09-17T17:29:46.652 INFO:tasks.rados.rados.0.smithi102.stdout:2: write initial oid 2 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:2: seq_num 1 ranges {0=32768} 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:2: writing smithi10238019-2 from 0 to 32768 tid 1 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:3: write initial oid 3 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:3: seq_num 2 ranges {0=49152} 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:3: writing smithi10238019-3 from 0 to 49152 tid 1 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:4: write initial oid 4 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:4: seq_num 3 ranges {0=65536} 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:4: writing smithi10238019-4 from 0 to 65536 tid 1 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:5: write initial oid 5 2024-09-17T17:29:46.653 INFO:tasks.rados.rados.0.smithi102.stdout:5: seq_num 4 ranges {0=81920} 2024-09-17T17:29:46.654 INFO:tasks.rados.rados.0.smithi102.stdout:5: writing smithi10238019-5 from 0 to 81920 tid 1 2024-09-17T17:29:46.654 INFO:tasks.rados.rados.0.smithi102.stdout:6: write initial oid 6 2024-09-17T17:29:46.654 INFO:tasks.rados.rados.0.smithi102.stdout:6: seq_num 5 ranges {0=98304} 2024-09-17T17:29:46.654 INFO:tasks.rados.rados.0.smithi102.stdout:6: writing smithi10238019-6 from 0 to 98304 tid 1 2024-09-17T17:29:46.654 INFO:tasks.rados.rados.0.smithi102.stdout:7: write initial oid 7 2024-09-17T17:29:46.655 INFO:tasks.rados.rados.0.smithi102.stdout:7: seq_num 6 ranges {0=114688} 2024-09-17T17:29:46.655 INFO:tasks.rados.rados.0.smithi102.stdout:7: writing smithi10238019-7 from 0 to 114688 tid 1 2024-09-17T17:29:46.655 INFO:tasks.rados.rados.0.smithi102.stdout:8: write initial oid 8 2024-09-17T17:29:46.655 INFO:tasks.rados.rados.0.smithi102.stdout:8: seq_num 7 ranges {0=131072} 2024-09-17T17:29:46.655 INFO:tasks.rados.rados.0.smithi102.stdout:8: writing smithi10238019-8 from 0 to 131072 tid 1 2024-09-17T17:29:46.655 INFO:tasks.rados.rados.0.smithi102.stdout:9: write initial oid 9 2024-09-17T17:29:46.656 INFO:tasks.rados.rados.0.smithi102.stdout:9: seq_num 8 ranges {0=147456} 2024-09-17T17:29:46.656 INFO:tasks.rados.rados.0.smithi102.stdout:9: writing smithi10238019-9 from 0 to 147456 tid 1 2024-09-17T17:29:46.656 INFO:tasks.rados.rados.0.smithi102.stdout:10: write initial oid 10 2024-09-17T17:29:46.656 INFO:tasks.rados.rados.0.smithi102.stdout:10: seq_num 9 ranges {0=163840} 2024-09-17T17:29:46.658 INFO:tasks.rados.rados.0.smithi102.stdout:10: writing smithi10238019-10 from 0 to 163840 tid 1 2024-09-17T17:29:46.658 INFO:tasks.rados.rados.0.smithi102.stdout:11: write initial oid 11 2024-09-17T17:29:46.658 INFO:tasks.rados.rados.0.smithi102.stdout:11: seq_num 10 ranges {0=180224} 2024-09-17T17:29:46.659 INFO:tasks.rados.rados.0.smithi102.stdout:11: writing smithi10238019-11 from 0 to 180224 tid 1 2024-09-17T17:29:46.659 INFO:tasks.rados.rados.0.smithi102.stdout:12: write initial oid 12 2024-09-17T17:29:46.659 INFO:tasks.rados.rados.0.smithi102.stdout:12: seq_num 11 ranges {0=196608} 2024-09-17T17:29:46.661 INFO:tasks.rados.rados.0.smithi102.stdout:12: writing smithi10238019-12 from 0 to 196608 tid 1 2024-09-17T17:29:46.661 INFO:tasks.rados.rados.0.smithi102.stdout:13: write initial oid 13 2024-09-17T17:29:46.661 INFO:tasks.rados.rados.0.smithi102.stdout:13: seq_num 12 ranges {0=212992} 2024-09-17T17:29:46.662 INFO:tasks.rados.rados.0.smithi102.stdout:13: writing smithi10238019-13 from 0 to 212992 tid 1 2024-09-17T17:29:46.663 INFO:tasks.rados.rados.0.smithi102.stdout:14: write initial oid 14 2024-09-17T17:29:46.663 INFO:tasks.rados.rados.0.smithi102.stdout:14: seq_num 13 ranges {0=229376} 2024-09-17T17:29:46.664 INFO:tasks.rados.rados.0.smithi102.stdout:14: writing smithi10238019-14 from 0 to 229376 tid 1 2024-09-17T17:29:46.665 INFO:tasks.rados.rados.0.smithi102.stdout:15: write initial oid 15 2024-09-17T17:29:46.665 INFO:tasks.rados.rados.0.smithi102.stdout:15: seq_num 14 ranges {0=245760} 2024-09-17T17:29:46.666 INFO:tasks.rados.rados.0.smithi102.stdout:15: writing smithi10238019-15 from 0 to 245760 tid 1 2024-09-17T17:29:46.667 INFO:tasks.rados.rados.0.smithi102.stdout:16: write initial oid 16 2024-09-17T17:29:46.667 INFO:tasks.rados.rados.0.smithi102.stdout:16: seq_num 15 ranges {0=262144} 2024-09-17T17:29:46.669 INFO:tasks.rados.rados.0.smithi102.stdout:16: writing smithi10238019-16 from 0 to 262144 tid 1 2024-09-17T17:29:46.669 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.672 INFO:tasks.rados.rados.0.smithi102.stdout:11: finishing write tid 1 to smithi10238019-11 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: finishing write tid 1 to smithi10238019-2 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: finishing write tid 2 to smithi10238019-2 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: finishing write tid 3 to smithi10238019-2 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: oid 2 updating version 0 to 1 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: oid 2 updating version 1 to 2 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:2: done (15 left) 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:17: write initial oid 17 2024-09-17T17:29:46.674 INFO:tasks.rados.rados.0.smithi102.stdout:17: seq_num 16 ranges {0=278528} 2024-09-17T17:29:46.676 INFO:tasks.rados.rados.0.smithi102.stdout:17: writing smithi10238019-17 from 0 to 278528 tid 1 2024-09-17T17:29:46.676 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.676 INFO:tasks.rados.rados.0.smithi102.stdout:9: finishing write tid 1 to smithi10238019-9 2024-09-17T17:29:46.676 INFO:tasks.rados.rados.0.smithi102.stdout:9: finishing write tid 2 to smithi10238019-9 2024-09-17T17:29:46.676 INFO:tasks.rados.rados.0.smithi102.stdout:9: finishing write tid 3 to smithi10238019-9 2024-09-17T17:29:46.676 INFO:tasks.rados.rados.0.smithi102.stdout:9: oid 9 updating version 0 to 1 2024-09-17T17:29:46.677 INFO:tasks.rados.rados.0.smithi102.stdout:9: oid 9 updating version 1 to 2 2024-09-17T17:29:46.677 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-17T17:29:46.677 INFO:tasks.rados.rados.0.smithi102.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-17T17:29:46.677 INFO:tasks.rados.rados.0.smithi102.stdout:9: done (15 left) 2024-09-17T17:29:46.677 INFO:tasks.rados.rados.0.smithi102.stdout:18: write initial oid 18 2024-09-17T17:29:46.677 INFO:tasks.rados.rados.0.smithi102.stdout:18: seq_num 17 ranges {0=294912} 2024-09-17T17:29:46.678 INFO:tasks.rados.rados.0.smithi102.stdout:18: writing smithi10238019-18 from 0 to 294912 tid 1 2024-09-17T17:29:46.678 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.679 INFO:tasks.rados.rados.0.smithi102.stdout:15: finishing write tid 1 to smithi10238019-15 2024-09-17T17:29:46.683 INFO:tasks.rados.rados.0.smithi102.stdout:15: finishing write tid 2 to smithi10238019-15 2024-09-17T17:29:46.685 INFO:tasks.rados.rados.0.smithi102.stdout:6: finishing write tid 1 to smithi10238019-6 2024-09-17T17:29:46.685 INFO:tasks.rados.rados.0.smithi102.stdout:6: finishing write tid 2 to smithi10238019-6 2024-09-17T17:29:46.685 INFO:tasks.rados.rados.0.smithi102.stdout:12: finishing write tid 1 to smithi10238019-12 2024-09-17T17:29:46.686 INFO:tasks.rados.rados.0.smithi102.stdout:12: finishing write tid 2 to smithi10238019-12 2024-09-17T17:29:46.687 INFO:tasks.rados.rados.0.smithi102.stdout:6: finishing write tid 3 to smithi10238019-6 2024-09-17T17:29:46.687 INFO:tasks.rados.rados.0.smithi102.stdout:6: oid 6 updating version 0 to 3 2024-09-17T17:29:46.687 INFO:tasks.rados.rados.0.smithi102.stdout:6: oid 6 updating version 3 to 4 2024-09-17T17:29:46.687 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 6 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-17T17:29:46.687 INFO:tasks.rados.rados.0.smithi102.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-17T17:29:46.687 INFO:tasks.rados.rados.0.smithi102.stdout:6: done (15 left) 2024-09-17T17:29:46.688 INFO:tasks.rados.rados.0.smithi102.stdout:19: write initial oid 19 2024-09-17T17:29:46.688 INFO:tasks.rados.rados.0.smithi102.stdout:19: seq_num 18 ranges {0=311296} 2024-09-17T17:29:46.689 INFO:tasks.rados.rados.0.smithi102.stdout:19: writing smithi10238019-19 from 0 to 311296 tid 1 2024-09-17T17:29:46.689 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.689 INFO:tasks.rados.rados.0.smithi102.stdout:12: finishing write tid 3 to smithi10238019-12 2024-09-17T17:29:46.689 INFO:tasks.rados.rados.0.smithi102.stdout:12: oid 12 updating version 0 to 5 2024-09-17T17:29:46.689 INFO:tasks.rados.rados.0.smithi102.stdout:12: oid 12 updating version 5 to 6 2024-09-17T17:29:46.690 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-17T17:29:46.690 INFO:tasks.rados.rados.0.smithi102.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-17T17:29:46.690 INFO:tasks.rados.rados.0.smithi102.stdout:12: done (15 left) 2024-09-17T17:29:46.690 INFO:tasks.rados.rados.0.smithi102.stdout:20: write initial oid 20 2024-09-17T17:29:46.690 INFO:tasks.rados.rados.0.smithi102.stdout:20: seq_num 19 ranges {0=327680} 2024-09-17T17:29:46.691 INFO:tasks.rados.rados.0.smithi102.stdout:20: writing smithi10238019-20 from 0 to 327680 tid 1 2024-09-17T17:29:46.691 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.691 INFO:tasks.rados.rados.0.smithi102.stdout:18: finishing write tid 1 to smithi10238019-18 2024-09-17T17:29:46.691 INFO:tasks.rados.rados.0.smithi102.stdout:18: finishing write tid 2 to smithi10238019-18 2024-09-17T17:29:46.692 INFO:tasks.rados.rados.0.smithi102.stdout:18: finishing write tid 3 to smithi10238019-18 2024-09-17T17:29:46.692 INFO:tasks.rados.rados.0.smithi102.stdout:18: oid 18 updating version 0 to 8 2024-09-17T17:29:46.692 INFO:tasks.rados.rados.0.smithi102.stdout:18: oid 18 version 8 is already newer than 7 2024-09-17T17:29:46.692 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 18 v 8 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-17T17:29:46.692 INFO:tasks.rados.rados.0.smithi102.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-17T17:29:46.693 INFO:tasks.rados.rados.0.smithi102.stdout:18: done (15 left) 2024-09-17T17:29:46.693 INFO:tasks.rados.rados.0.smithi102.stdout:21: write initial oid 21 2024-09-17T17:29:46.693 INFO:tasks.rados.rados.0.smithi102.stdout:21: seq_num 20 ranges {0=344064} 2024-09-17T17:29:46.694 INFO:tasks.rados.rados.0.smithi102.stdout:21: writing smithi10238019-21 from 0 to 344064 tid 1 2024-09-17T17:29:46.694 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:17: finishing write tid 1 to smithi10238019-17 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:17: finishing write tid 2 to smithi10238019-17 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:15: finishing write tid 3 to smithi10238019-15 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:15: oid 15 updating version 0 to 1 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:15: oid 15 updating version 1 to 2 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:15: done (15 left) 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:22: write initial oid 22 2024-09-17T17:29:46.695 INFO:tasks.rados.rados.0.smithi102.stdout:22: seq_num 21 ranges {0=360448} 2024-09-17T17:29:46.696 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:46.692+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:22: writing smithi10238019-22 from 0 to 360448 tid 1 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:17: finishing write tid 3 to smithi10238019-17 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:17: oid 17 updating version 0 to 4 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:17: oid 17 version 4 is already newer than 3 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:17: done (15 left) 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:23: write initial oid 23 2024-09-17T17:29:46.697 INFO:tasks.rados.rados.0.smithi102.stdout:23: seq_num 22 ranges {0=376832} 2024-09-17T17:29:46.698 INFO:tasks.rados.rados.0.smithi102.stdout:23: writing smithi10238019-23 from 0 to 376832 tid 1 2024-09-17T17:29:46.698 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.699 INFO:tasks.rados.rados.0.smithi102.stdout:22: finishing write tid 1 to smithi10238019-22 2024-09-17T17:29:46.701 INFO:tasks.rados.rados.0.smithi102.stdout:22: finishing write tid 2 to smithi10238019-22 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:22: finishing write tid 3 to smithi10238019-22 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:22: oid 22 updating version 0 to 1 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:22: oid 22 updating version 1 to 2 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 22 v 2 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:22: done (15 left) 2024-09-17T17:29:46.702 INFO:tasks.rados.rados.0.smithi102.stdout:24: write initial oid 24 2024-09-17T17:29:46.703 INFO:tasks.rados.rados.0.smithi102.stdout:24: seq_num 23 ranges {0=393216} 2024-09-17T17:29:46.704 INFO:tasks.rados.rados.0.smithi102.stdout:24: writing smithi10238019-24 from 0 to 393216 tid 1 2024-09-17T17:29:46.704 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.710 INFO:tasks.rados.rados.0.smithi102.stdout:23: finishing write tid 1 to smithi10238019-23 2024-09-17T17:29:46.710 INFO:tasks.rados.rados.0.smithi102.stdout:23: finishing write tid 2 to smithi10238019-23 2024-09-17T17:29:46.710 INFO:tasks.rados.rados.0.smithi102.stdout:24: finishing write tid 1 to smithi10238019-24 2024-09-17T17:29:46.710 INFO:tasks.rados.rados.0.smithi102.stdout:24: finishing write tid 2 to smithi10238019-24 2024-09-17T17:29:46.710 INFO:tasks.rados.rados.0.smithi102.stdout:23: finishing write tid 3 to smithi10238019-23 2024-09-17T17:29:46.710 INFO:tasks.rados.rados.0.smithi102.stdout:23: oid 23 updating version 0 to 2 2024-09-17T17:29:46.711 INFO:tasks.rados.rados.0.smithi102.stdout:23: oid 23 version 2 is already newer than 1 2024-09-17T17:29:46.711 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 23 v 2 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-17T17:29:46.711 INFO:tasks.rados.rados.0.smithi102.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-17T17:29:46.711 INFO:tasks.rados.rados.0.smithi102.stdout:23: done (15 left) 2024-09-17T17:29:46.711 INFO:tasks.rados.rados.0.smithi102.stdout:25: write initial oid 25 2024-09-17T17:29:46.711 INFO:tasks.rados.rados.0.smithi102.stdout:25: seq_num 24 ranges {0=409600} 2024-09-17T17:29:46.712 INFO:tasks.rados.rados.0.smithi102.stdout:25: writing smithi10238019-25 from 0 to 409600 tid 1 2024-09-17T17:29:46.712 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.712 INFO:tasks.rados.rados.0.smithi102.stdout:24: finishing write tid 3 to smithi10238019-24 2024-09-17T17:29:46.712 INFO:tasks.rados.rados.0.smithi102.stdout:24: oid 24 updating version 0 to 1 2024-09-17T17:29:46.713 INFO:tasks.rados.rados.0.smithi102.stdout:24: oid 24 updating version 1 to 2 2024-09-17T17:29:46.713 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 24 v 2 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-17T17:29:46.713 INFO:tasks.rados.rados.0.smithi102.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-17T17:29:46.713 INFO:tasks.rados.rados.0.smithi102.stdout:24: done (15 left) 2024-09-17T17:29:46.713 INFO:tasks.rados.rados.0.smithi102.stdout:26: write initial oid 26 2024-09-17T17:29:46.713 INFO:tasks.rados.rados.0.smithi102.stdout:26: seq_num 25 ranges {0=425984} 2024-09-17T17:29:46.715 INFO:tasks.rados.rados.0.smithi102.stdout:26: writing smithi10238019-26 from 0 to 425984 tid 1 2024-09-17T17:29:46.715 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:11: finishing write tid 2 to smithi10238019-11 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:25: finishing write tid 1 to smithi10238019-25 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:11: finishing write tid 3 to smithi10238019-11 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:11: oid 11 updating version 0 to 1 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:11: oid 11 updating version 1 to 2 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 11 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-17T17:29:46.717 INFO:tasks.rados.rados.0.smithi102.stdout:11: done (15 left) 2024-09-17T17:29:46.718 INFO:tasks.rados.rados.0.smithi102.stdout:27: write initial oid 27 2024-09-17T17:29:46.718 INFO:tasks.rados.rados.0.smithi102.stdout:27: seq_num 26 ranges {0=442368} 2024-09-17T17:29:46.719 INFO:tasks.rados.rados.0.smithi102.stdout:27: writing smithi10238019-27 from 0 to 442368 tid 1 2024-09-17T17:29:46.719 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:46.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:29:46.748 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:46.748 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:46.757 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:29:46.796 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:46.794+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:46.897 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:46.895+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:46.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:29:46.997 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:46.994+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:47.011 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.011 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.020 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:29:47.103 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:47.101+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:47.162 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.162 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:29:47.198 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:47.196+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:47.278 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.278 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:29:47.299 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:47.297+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:47.393 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.393 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.400 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:47.398+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:47.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:29:47.489 INFO:tasks.rados.rados.0.smithi102.stdout:25: finishing write tid 2 to smithi10238019-25 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:25: finishing write tid 3 to smithi10238019-25 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:25: oid 25 updating version 0 to 3 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:25: oid 25 updating version 3 to 4 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 25 v 4 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:25: done (15 left) 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:28: write initial oid 28 2024-09-17T17:29:47.490 INFO:tasks.rados.rados.0.smithi102.stdout:28: seq_num 27 ranges {0=458752} 2024-09-17T17:29:47.492 INFO:tasks.rados.rados.0.smithi102.stdout:28: writing smithi10238019-28 from 0 to 458752 tid 1 2024-09-17T17:29:47.492 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:47.500 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:47.498+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:47.509 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.509 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:29:47.602 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:47.600+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:47.625 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.626 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:29:47.652 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:29:47.703 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:47.701+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:47.745 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.745 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:29:47.803 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:47.801+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:47.861 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:47.861 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:47.869 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:29:47.904 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:47.902+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:48.005 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:48.003+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:29:48.018 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:48.018 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:48.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:29:48.106 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:48.103+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:48.173 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:48.174 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:48.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:29:48.196 INFO:tasks.rados.rados.0.smithi102.stdout:5: finishing write tid 1 to smithi10238019-5 2024-09-17T17:29:48.200 INFO:tasks.rados.rados.0.smithi102.stdout:21: finishing write tid 1 to smithi10238019-21 2024-09-17T17:29:48.200 INFO:tasks.rados.rados.0.smithi102.stdout:4: finishing write tid 1 to smithi10238019-4 2024-09-17T17:29:48.203 INFO:tasks.rados.rados.0.smithi102.stdout:21: finishing write tid 2 to smithi10238019-21 2024-09-17T17:29:48.203 INFO:tasks.rados.rados.0.smithi102.stdout:7: finishing write tid 1 to smithi10238019-7 2024-09-17T17:29:48.203 INFO:tasks.rados.rados.0.smithi102.stdout:4: finishing write tid 2 to smithi10238019-4 2024-09-17T17:29:48.203 INFO:tasks.rados.rados.0.smithi102.stdout:7: finishing write tid 2 to smithi10238019-7 2024-09-17T17:29:48.203 INFO:tasks.rados.rados.0.smithi102.stdout:16: finishing write tid 1 to smithi10238019-16 2024-09-17T17:29:48.204 INFO:tasks.rados.rados.0.smithi102.stdout:16: finishing write tid 2 to smithi10238019-16 2024-09-17T17:29:48.204 INFO:tasks.rados.rados.0.smithi102.stdout:21: finishing write tid 3 to smithi10238019-21 2024-09-17T17:29:48.204 INFO:tasks.rados.rados.0.smithi102.stdout:21: oid 21 updating version 0 to 1 2024-09-17T17:29:48.205 INFO:tasks.rados.rados.0.smithi102.stdout:21: oid 21 updating version 1 to 2 2024-09-17T17:29:48.205 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 21 v 2 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-17T17:29:48.205 INFO:tasks.rados.rados.0.smithi102.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-17T17:29:48.205 INFO:tasks.rados.rados.0.smithi102.stdout:21: done (15 left) 2024-09-17T17:29:48.205 INFO:tasks.rados.rados.0.smithi102.stdout:29: write initial oid 29 2024-09-17T17:29:48.205 INFO:tasks.rados.rados.0.smithi102.stdout:29: seq_num 28 ranges {0=475136} 2024-09-17T17:29:48.206 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:48.205+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:48.208 INFO:tasks.rados.rados.0.smithi102.stdout:29: writing smithi10238019-29 from 0 to 475136 tid 1 2024-09-17T17:29:48.208 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.208 INFO:tasks.rados.rados.0.smithi102.stdout:7: finishing write tid 3 to smithi10238019-7 2024-09-17T17:29:48.208 INFO:tasks.rados.rados.0.smithi102.stdout:7: oid 7 updating version 0 to 1 2024-09-17T17:29:48.208 INFO:tasks.rados.rados.0.smithi102.stdout:7: oid 7 updating version 1 to 2 2024-09-17T17:29:48.208 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:4: finishing write tid 3 to smithi10238019-4 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:4: oid 4 updating version 0 to 1 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:4: oid 4 updating version 1 to 2 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:16: finishing write tid 3 to smithi10238019-16 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:16: oid 16 updating version 0 to 4 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:16: oid 16 version 4 is already newer than 3 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 4 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-17T17:29:48.209 INFO:tasks.rados.rados.0.smithi102.stdout:4: done (15 left) 2024-09-17T17:29:48.210 INFO:tasks.rados.rados.0.smithi102.stdout:7: done (14 left) 2024-09-17T17:29:48.210 INFO:tasks.rados.rados.0.smithi102.stdout:16: done (13 left) 2024-09-17T17:29:48.210 INFO:tasks.rados.rados.0.smithi102.stdout:30: write initial oid 30 2024-09-17T17:29:48.210 INFO:tasks.rados.rados.0.smithi102.stdout:30: seq_num 29 ranges {0=491520} 2024-09-17T17:29:48.210 INFO:tasks.rados.rados.0.smithi102.stdout:30: writing smithi10238019-30 from 0 to 491520 tid 1 2024-09-17T17:29:48.211 INFO:tasks.rados.rados.0.smithi102.stdout:31: write initial oid 31 2024-09-17T17:29:48.211 INFO:tasks.rados.rados.0.smithi102.stdout:31: seq_num 30 ranges {0=507904} 2024-09-17T17:29:48.213 INFO:tasks.rados.rados.0.smithi102.stdout:31: writing smithi10238019-31 from 0 to 507904 tid 1 2024-09-17T17:29:48.213 INFO:tasks.rados.rados.0.smithi102.stdout:32: write initial oid 32 2024-09-17T17:29:48.213 INFO:tasks.rados.rados.0.smithi102.stdout:32: seq_num 31 ranges {0=524288} 2024-09-17T17:29:48.216 INFO:tasks.rados.rados.0.smithi102.stdout:32: writing smithi10238019-32 from 0 to 524288 tid 1 2024-09-17T17:29:48.216 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.218 INFO:tasks.rados.rados.0.smithi102.stdout:31: finishing write tid 1 to smithi10238019-31 2024-09-17T17:29:48.219 INFO:tasks.rados.rados.0.smithi102.stdout:31: finishing write tid 2 to smithi10238019-31 2024-09-17T17:29:48.220 INFO:tasks.rados.rados.0.smithi102.stdout:31: finishing write tid 3 to smithi10238019-31 2024-09-17T17:29:48.220 INFO:tasks.rados.rados.0.smithi102.stdout:31: oid 31 updating version 0 to 3 2024-09-17T17:29:48.221 INFO:tasks.rados.rados.0.smithi102.stdout:31: oid 31 updating version 3 to 4 2024-09-17T17:29:48.221 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 4 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-17T17:29:48.221 INFO:tasks.rados.rados.0.smithi102.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-17T17:29:48.221 INFO:tasks.rados.rados.0.smithi102.stdout:31: done (15 left) 2024-09-17T17:29:48.221 INFO:tasks.rados.rados.0.smithi102.stdout:33: write initial oid 33 2024-09-17T17:29:48.221 INFO:tasks.rados.rados.0.smithi102.stdout:33: seq_num 32 ranges {0=540672} 2024-09-17T17:29:48.223 INFO:tasks.rados.rados.0.smithi102.stdout:33: writing smithi10238019-33 from 0 to 540672 tid 1 2024-09-17T17:29:48.223 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.239 INFO:tasks.rados.rados.0.smithi102.stdout:33: finishing write tid 1 to smithi10238019-33 2024-09-17T17:29:48.239 INFO:tasks.rados.rados.0.smithi102.stdout:33: finishing write tid 2 to smithi10238019-33 2024-09-17T17:29:48.240 INFO:tasks.rados.rados.0.smithi102.stdout:33: finishing write tid 3 to smithi10238019-33 2024-09-17T17:29:48.240 INFO:tasks.rados.rados.0.smithi102.stdout:33: oid 33 updating version 0 to 1 2024-09-17T17:29:48.240 INFO:tasks.rados.rados.0.smithi102.stdout:33: oid 33 updating version 1 to 2 2024-09-17T17:29:48.241 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 33 v 2 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-17T17:29:48.241 INFO:tasks.rados.rados.0.smithi102.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-17T17:29:48.241 INFO:tasks.rados.rados.0.smithi102.stdout:33: done (15 left) 2024-09-17T17:29:48.241 INFO:tasks.rados.rados.0.smithi102.stdout:34: write initial oid 34 2024-09-17T17:29:48.241 INFO:tasks.rados.rados.0.smithi102.stdout:34: seq_num 33 ranges {0=557056} 2024-09-17T17:29:48.243 INFO:tasks.rados.rados.0.smithi102.stdout:34: writing smithi10238019-34 from 0 to 557056 tid 1 2024-09-17T17:29:48.243 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.247 INFO:tasks.rados.rados.0.smithi102.stdout:34: finishing write tid 1 to smithi10238019-34 2024-09-17T17:29:48.298 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:48.298 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:48.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:29:48.307 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:48.305+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:48.402 INFO:tasks.rados.rados.0.smithi102.stdout:5: finishing write tid 2 to smithi10238019-5 2024-09-17T17:29:48.404 INFO:tasks.rados.rados.0.smithi102.stdout:14: finishing write tid 1 to smithi10238019-14 2024-09-17T17:29:48.404 INFO:tasks.rados.rados.0.smithi102.stdout:14: finishing write tid 2 to smithi10238019-14 2024-09-17T17:29:48.404 INFO:tasks.rados.rados.0.smithi102.stdout:5: finishing write tid 3 to smithi10238019-5 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:5: oid 5 updating version 0 to 1 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:5: oid 5 updating version 1 to 2 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:5: done (15 left) 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:35: write initial oid 35 2024-09-17T17:29:48.405 INFO:tasks.rados.rados.0.smithi102.stdout:35: seq_num 34 ranges {0=573440} 2024-09-17T17:29:48.407 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:48.404+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:48.408 INFO:tasks.rados.rados.0.smithi102.stdout:35: writing smithi10238019-35 from 0 to 573440 tid 1 2024-09-17T17:29:48.408 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.408 INFO:tasks.rados.rados.0.smithi102.stdout:14: finishing write tid 3 to smithi10238019-14 2024-09-17T17:29:48.408 INFO:tasks.rados.rados.0.smithi102.stdout:14: oid 14 updating version 0 to 2 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:14: oid 14 version 2 is already newer than 1 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 14 v 2 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:34: finishing write tid 2 to smithi10238019-34 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:20: finishing write tid 1 to smithi10238019-20 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:20: finishing write tid 2 to smithi10238019-20 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:14: done (15 left) 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:36: write initial oid 36 2024-09-17T17:29:48.409 INFO:tasks.rados.rados.0.smithi102.stdout:36: seq_num 35 ranges {0=589824} 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout:36: writing smithi10238019-36 from 0 to 589824 tid 1 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout:34: finishing write tid 3 to smithi10238019-34 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout:34: oid 34 updating version 0 to 6 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout:34: oid 34 version 6 is already newer than 5 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-17T17:29:48.412 INFO:tasks.rados.rados.0.smithi102.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:20: finishing write tid 3 to smithi10238019-20 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:20: oid 20 updating version 0 to 4 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:20: oid 20 version 4 is already newer than 3 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 20 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:29: finishing write tid 1 to smithi10238019-29 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:29: finishing write tid 2 to smithi10238019-29 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:20: done (15 left) 2024-09-17T17:29:48.413 INFO:tasks.rados.rados.0.smithi102.stdout:34: done (14 left) 2024-09-17T17:29:48.414 INFO:tasks.rados.rados.0.smithi102.stdout:37: write initial oid 37 2024-09-17T17:29:48.414 INFO:tasks.rados.rados.0.smithi102.stdout:37: seq_num 36 ranges {0=606208} 2024-09-17T17:29:48.416 INFO:tasks.rados.rados.0.smithi102.stdout:37: writing smithi10238019-37 from 0 to 606208 tid 1 2024-09-17T17:29:48.416 INFO:tasks.rados.rados.0.smithi102.stdout:38: write initial oid 38 2024-09-17T17:29:48.416 INFO:tasks.rados.rados.0.smithi102.stdout:38: seq_num 37 ranges {0=622592} 2024-09-17T17:29:48.419 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:48.419 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:48.420 INFO:tasks.rados.rados.0.smithi102.stdout:38: writing smithi10238019-38 from 0 to 622592 tid 1 2024-09-17T17:29:48.420 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.420 INFO:tasks.rados.rados.0.smithi102.stdout:29: finishing write tid 3 to smithi10238019-29 2024-09-17T17:29:48.420 INFO:tasks.rados.rados.0.smithi102.stdout:29: oid 29 updating version 0 to 6 2024-09-17T17:29:48.420 INFO:tasks.rados.rados.0.smithi102.stdout:29: oid 29 version 6 is already newer than 5 2024-09-17T17:29:48.420 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-17T17:29:48.421 INFO:tasks.rados.rados.0.smithi102.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-17T17:29:48.421 INFO:tasks.rados.rados.0.smithi102.stdout:29: done (15 left) 2024-09-17T17:29:48.421 INFO:tasks.rados.rados.0.smithi102.stdout:39: write initial oid 39 2024-09-17T17:29:48.421 INFO:tasks.rados.rados.0.smithi102.stdout:39: seq_num 38 ranges {0=638976} 2024-09-17T17:29:48.424 INFO:tasks.rados.rados.0.smithi102.stdout:39: writing smithi10238019-39 from 0 to 638976 tid 1 2024-09-17T17:29:48.424 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:35: finishing write tid 1 to smithi10238019-35 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:35: finishing write tid 2 to smithi10238019-35 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:35: finishing write tid 3 to smithi10238019-35 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:35: oid 35 updating version 0 to 7 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:35: oid 35 updating version 7 to 8 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 35 v 8 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-17T17:29:48.425 INFO:tasks.rados.rados.0.smithi102.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-17T17:29:48.426 INFO:tasks.rados.rados.0.smithi102.stdout:36: finishing write tid 1 to smithi10238019-36 2024-09-17T17:29:48.426 INFO:tasks.rados.rados.0.smithi102.stdout:35: done (15 left) 2024-09-17T17:29:48.426 INFO:tasks.rados.rados.0.smithi102.stdout:40: write initial oid 40 2024-09-17T17:29:48.426 INFO:tasks.rados.rados.0.smithi102.stdout:40: seq_num 39 ranges {0=671744} 2024-09-17T17:29:48.430 INFO:tasks.rados.rados.0.smithi102.stdout:40: writing smithi10238019-40 from 0 to 671744 tid 1 2024-09-17T17:29:48.430 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.430 INFO:tasks.rados.rados.0.smithi102.stdout:3: finishing write tid 1 to smithi10238019-3 2024-09-17T17:29:48.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:29:48.432 INFO:tasks.rados.rados.0.smithi102.stdout:37: finishing write tid 1 to smithi10238019-37 2024-09-17T17:29:48.432 INFO:tasks.rados.rados.0.smithi102.stdout:37: finishing write tid 2 to smithi10238019-37 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:37: finishing write tid 3 to smithi10238019-37 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:37: oid 37 updating version 0 to 5 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:37: oid 37 updating version 5 to 6 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:37: done (15 left) 2024-09-17T17:29:48.433 INFO:tasks.rados.rados.0.smithi102.stdout:41: write initial oid 41 2024-09-17T17:29:48.434 INFO:tasks.rados.rados.0.smithi102.stdout:41: seq_num 40 ranges {0=688128} 2024-09-17T17:29:48.438 INFO:tasks.rados.rados.0.smithi102.stdout:41: writing smithi10238019-41 from 0 to 688128 tid 1 2024-09-17T17:29:48.438 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.438 INFO:tasks.rados.rados.0.smithi102.stdout:1: finishing write tid 1 to smithi10238019-1 2024-09-17T17:29:48.438 INFO:tasks.rados.rados.0.smithi102.stdout:3: finishing write tid 2 to smithi10238019-3 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:13: finishing write tid 1 to smithi10238019-13 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:13: finishing write tid 2 to smithi10238019-13 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: finishing write tid 1 to smithi10238019-39 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:19: finishing write tid 1 to smithi10238019-19 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: finishing write tid 2 to smithi10238019-39 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:1: finishing write tid 2 to smithi10238019-1 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:8: finishing write tid 1 to smithi10238019-8 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:19: finishing write tid 2 to smithi10238019-19 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: finishing write tid 3 to smithi10238019-39 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: oid 39 updating version 0 to 4 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: oid 39 version 4 is already newer than 3 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 39 v 4 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-17T17:29:48.439 INFO:tasks.rados.rados.0.smithi102.stdout:39: done (15 left) 2024-09-17T17:29:48.440 INFO:tasks.rados.rados.0.smithi102.stdout:42: write initial oid 42 2024-09-17T17:29:48.440 INFO:tasks.rados.rados.0.smithi102.stdout:42: seq_num 41 ranges {0=704512} 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:42: writing smithi10238019-42 from 0 to 704512 tid 1 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:3: finishing write tid 3 to smithi10238019-3 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:3: oid 3 updating version 0 to 2 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:3: oid 3 updating version 2 to 3 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:1: finishing write tid 3 to smithi10238019-1 2024-09-17T17:29:48.444 INFO:tasks.rados.rados.0.smithi102.stdout:1: oid 1 updating version 0 to 2 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:1: oid 1 updating version 2 to 3 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:19: finishing write tid 3 to smithi10238019-19 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:19: oid 19 updating version 0 to 4 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:19: oid 19 updating version 4 to 5 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 19 v 5 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:27: finishing write tid 1 to smithi10238019-27 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:27: finishing write tid 2 to smithi10238019-27 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:13: finishing write tid 3 to smithi10238019-13 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:13: oid 13 updating version 0 to 1 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:13: oid 13 updating version 1 to 2 2024-09-17T17:29:48.445 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 13 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:32: finishing write tid 1 to smithi10238019-32 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:32: finishing write tid 2 to smithi10238019-32 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:1: done (15 left) 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:3: done (14 left) 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:13: done (13 left) 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:19: done (12 left) 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:43: write initial oid 43 2024-09-17T17:29:48.446 INFO:tasks.rados.rados.0.smithi102.stdout:43: seq_num 42 ranges {0=720896} 2024-09-17T17:29:48.447 INFO:tasks.rados.rados.0.smithi102.stdout:43: writing smithi10238019-43 from 0 to 720896 tid 1 2024-09-17T17:29:48.447 INFO:tasks.rados.rados.0.smithi102.stdout:44: write initial oid 44 2024-09-17T17:29:48.447 INFO:tasks.rados.rados.0.smithi102.stdout:44: seq_num 43 ranges {0=737280} 2024-09-17T17:29:48.451 INFO:tasks.rados.rados.0.smithi102.stdout:44: writing smithi10238019-44 from 0 to 737280 tid 1 2024-09-17T17:29:48.451 INFO:tasks.rados.rados.0.smithi102.stdout:45: write initial oid 45 2024-09-17T17:29:48.451 INFO:tasks.rados.rados.0.smithi102.stdout:45: seq_num 44 ranges {0=753664} 2024-09-17T17:29:48.455 INFO:tasks.rados.rados.0.smithi102.stdout:45: writing smithi10238019-45 from 0 to 753664 tid 1 2024-09-17T17:29:48.455 INFO:tasks.rados.rados.0.smithi102.stdout:46: write initial oid 46 2024-09-17T17:29:48.456 INFO:tasks.rados.rados.0.smithi102.stdout:46: seq_num 45 ranges {0=770048} 2024-09-17T17:29:48.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:29:48.460 INFO:tasks.rados.rados.0.smithi102.stdout:46: writing smithi10238019-46 from 0 to 770048 tid 1 2024-09-17T17:29:48.460 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.460 INFO:tasks.rados.rados.0.smithi102.stdout:27: finishing write tid 3 to smithi10238019-27 2024-09-17T17:29:48.460 INFO:tasks.rados.rados.0.smithi102.stdout:27: oid 27 updating version 0 to 6 2024-09-17T17:29:48.460 INFO:tasks.rados.rados.0.smithi102.stdout:27: oid 27 updating version 6 to 7 2024-09-17T17:29:48.461 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 27 v 7 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-17T17:29:48.461 INFO:tasks.rados.rados.0.smithi102.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-17T17:29:48.461 INFO:tasks.rados.rados.0.smithi102.stdout:8: finishing write tid 2 to smithi10238019-8 2024-09-17T17:29:48.461 INFO:tasks.rados.rados.0.smithi102.stdout:27: done (15 left) 2024-09-17T17:29:48.461 INFO:tasks.rados.rados.0.smithi102.stdout:47: write initial oid 47 2024-09-17T17:29:48.461 INFO:tasks.rados.rados.0.smithi102.stdout:47: seq_num 46 ranges {0=786432} 2024-09-17T17:29:48.464 INFO:tasks.rados.rados.0.smithi102.stdout:47: writing smithi10238019-47 from 0 to 786432 tid 1 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:10: finishing write tid 1 to smithi10238019-10 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:10: finishing write tid 2 to smithi10238019-10 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:30: finishing write tid 1 to smithi10238019-30 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:40: finishing write tid 1 to smithi10238019-40 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:32: finishing write tid 3 to smithi10238019-32 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:32: oid 32 updating version 0 to 3 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:32: oid 32 updating version 3 to 4 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 32 v 4 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:40: finishing write tid 2 to smithi10238019-40 2024-09-17T17:29:48.465 INFO:tasks.rados.rados.0.smithi102.stdout:30: finishing write tid 2 to smithi10238019-30 2024-09-17T17:29:48.466 INFO:tasks.rados.rados.0.smithi102.stdout:32: done (15 left) 2024-09-17T17:29:48.466 INFO:tasks.rados.rados.0.smithi102.stdout:48: write initial oid 48 2024-09-17T17:29:48.466 INFO:tasks.rados.rados.0.smithi102.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-17T17:29:48.466 INFO:tasks.rados.rados.0.smithi102.stdout:48: writing smithi10238019-48 from 0 to 425984 tid 1 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:48: writing smithi10238019-48 from 425984 to 802816 tid 2 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 16 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:26: finishing write tid 1 to smithi10238019-26 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:10: finishing write tid 3 to smithi10238019-10 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:10: oid 10 updating version 0 to 6 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:10: oid 10 updating version 6 to 7 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 10 v 7 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:40: finishing write tid 3 to smithi10238019-40 2024-09-17T17:29:48.471 INFO:tasks.rados.rados.0.smithi102.stdout:40: oid 40 updating version 0 to 3 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:40: oid 40 updating version 3 to 4 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 40 v 4 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:26: finishing write tid 2 to smithi10238019-26 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:30: finishing write tid 3 to smithi10238019-30 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:30: oid 30 updating version 0 to 8 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:30: oid 30 updating version 8 to 9 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 30 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:8: finishing write tid 3 to smithi10238019-8 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:8: oid 8 updating version 0 to 4 2024-09-17T17:29:48.472 INFO:tasks.rados.rados.0.smithi102.stdout:8: oid 8 updating version 4 to 5 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 8 v 5 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:26: finishing write tid 3 to smithi10238019-26 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:26: oid 26 updating version 0 to 8 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:26: oid 26 updating version 8 to 9 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:42: finishing write tid 1 to smithi10238019-42 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:36: finishing write tid 2 to smithi10238019-36 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:28: finishing write tid 1 to smithi10238019-28 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:41: finishing write tid 1 to smithi10238019-41 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:28: finishing write tid 2 to smithi10238019-28 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:38: finishing write tid 1 to smithi10238019-38 2024-09-17T17:29:48.473 INFO:tasks.rados.rados.0.smithi102.stdout:36: finishing write tid 3 to smithi10238019-36 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:36: oid 36 updating version 0 to 7 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:36: oid 36 updating version 7 to 8 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 36 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:28: finishing write tid 3 to smithi10238019-28 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:28: oid 28 updating version 0 to 10 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:28: oid 28 updating version 10 to 11 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 28 v 11 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:43: finishing write tid 1 to smithi10238019-43 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:41: finishing write tid 2 to smithi10238019-41 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:43: finishing write tid 2 to smithi10238019-43 2024-09-17T17:29:48.474 INFO:tasks.rados.rados.0.smithi102.stdout:43: finishing write tid 3 to smithi10238019-43 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:43: oid 43 updating version 0 to 9 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:43: oid 43 updating version 9 to 10 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 43 v 10 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:41: finishing write tid 3 to smithi10238019-41 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:41: oid 41 updating version 0 to 3 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:41: oid 41 updating version 3 to 4 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 41 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:44: finishing write tid 1 to smithi10238019-44 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:38: finishing write tid 2 to smithi10238019-38 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:38: finishing write tid 3 to smithi10238019-38 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:38: oid 38 updating version 0 to 13 2024-09-17T17:29:48.475 INFO:tasks.rados.rados.0.smithi102.stdout:38: oid 38 version 13 is already newer than 12 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 38 v 13 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:45: finishing write tid 1 to smithi10238019-45 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:44: finishing write tid 2 to smithi10238019-44 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:44: finishing write tid 3 to smithi10238019-44 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:44: oid 44 updating version 0 to 4 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:44: oid 44 version 4 is already newer than 3 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 44 v 4 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:46: finishing write tid 1 to smithi10238019-46 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:46: finishing write tid 2 to smithi10238019-46 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:46: finishing write tid 3 to smithi10238019-46 2024-09-17T17:29:48.476 INFO:tasks.rados.rados.0.smithi102.stdout:46: oid 46 updating version 0 to 4 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:46: oid 46 version 4 is already newer than 3 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 4 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:8: done (15 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:10: done (14 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:26: done (13 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:28: done (12 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:30: done (11 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:36: done (10 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:38: done (9 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:40: done (8 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:41: done (7 left) 2024-09-17T17:29:48.477 INFO:tasks.rados.rados.0.smithi102.stdout:43: done (6 left) 2024-09-17T17:29:48.478 INFO:tasks.rados.rados.0.smithi102.stdout:44: done (5 left) 2024-09-17T17:29:48.478 INFO:tasks.rados.rados.0.smithi102.stdout:46: done (4 left) 2024-09-17T17:29:48.478 INFO:tasks.rados.rados.0.smithi102.stdout:49: write initial oid 49 2024-09-17T17:29:48.478 INFO:tasks.rados.rados.0.smithi102.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-17T17:29:48.479 INFO:tasks.rados.rados.0.smithi102.stdout:49: writing smithi10238019-49 from 0 to 442368 tid 1 2024-09-17T17:29:48.479 INFO:tasks.rados.rados.0.smithi102.stdout:49: writing smithi10238019-49 from 442368 to 819200 tid 2 2024-09-17T17:29:48.479 INFO:tasks.rados.rados.0.smithi102.stdout:50: write initial oid 50 2024-09-17T17:29:48.479 INFO:tasks.rados.rados.0.smithi102.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-17T17:29:48.479 INFO:tasks.rados.rados.0.smithi102.stdout:50: writing smithi10238019-50 from 0 to 458752 tid 1 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:50: writing smithi10238019-50 from 458752 to 835584 tid 2 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:51: rollback oid 34 current snap is 0 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:51: done (6 left) 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:52: copy_from oid 31 from oid 2 current snap is 0 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:48: finishing write tid 1 to smithi10238019-48 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:47: finishing write tid 1 to smithi10238019-47 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:45: finishing write tid 2 to smithi10238019-45 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:42: finishing write tid 2 to smithi10238019-42 2024-09-17T17:29:48.481 INFO:tasks.rados.rados.0.smithi102.stdout:53: read oid 14 snap -1 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:53: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:54: snap_create 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:45: finishing write tid 3 to smithi10238019-45 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:45: oid 45 updating version 0 to 15 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:45: oid 45 version 15 is already newer than 14 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:48: finishing write tid 2 to smithi10238019-48 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:48: finishing write tid 3 to smithi10238019-48 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:42: finishing write tid 3 to smithi10238019-42 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:42: oid 42 updating version 0 to 5 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:42: oid 42 updating version 5 to 6 2024-09-17T17:29:48.482 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:48: finishing write tid 4 to smithi10238019-48 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:48: oid 48 updating version 0 to 5 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:48: oid 48 updating version 5 to 6 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:48: oid 48 updating version 6 to 7 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 48 v 7 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:49: finishing write tid 1 to smithi10238019-49 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:47: finishing write tid 2 to smithi10238019-47 2024-09-17T17:29:48.483 INFO:tasks.rados.rados.0.smithi102.stdout:49: finishing write tid 2 to smithi10238019-49 2024-09-17T17:29:48.484 INFO:tasks.rados.rados.0.smithi102.stdout:47: finishing write tid 3 to smithi10238019-47 2024-09-17T17:29:48.484 INFO:tasks.rados.rados.0.smithi102.stdout:47: oid 47 updating version 0 to 10 2024-09-17T17:29:48.484 INFO:tasks.rados.rados.0.smithi102.stdout:47: oid 47 version 10 is already newer than 9 2024-09-17T17:29:48.484 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 47 v 10 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-17T17:29:48.484 INFO:tasks.rados.rados.0.smithi102.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-17T17:29:48.485 INFO:tasks.rados.rados.0.smithi102.stdout:50: finishing write tid 1 to smithi10238019-50 2024-09-17T17:29:48.485 INFO:tasks.rados.rados.0.smithi102.stdout:50: finishing write tid 2 to smithi10238019-50 2024-09-17T17:29:48.491 INFO:tasks.rados.rados.0.smithi102.stdout:52: finishing copy_from racing read to smithi10238019-31 2024-09-17T17:29:48.491 INFO:tasks.rados.rados.0.smithi102.stdout:52: finishing copy_from to smithi10238019-31 2024-09-17T17:29:48.492 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-17T17:29:48.508 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:48.506+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:48.550 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:48.550 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:48.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:29:48.609 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:48.606+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:48.615 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:48.616 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:48.638 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:48.639 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:48.661 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:42: done (8 left) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:45: done (7 left) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:47: done (6 left) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:48: done (5 left) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:52: done (4 left) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:54: done (3 left) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:55: read oid 18 snap -1 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:55: expect (ObjNum 17 snap 0 seq_num 17) 2024-09-17T17:29:48.665 INFO:tasks.rados.rados.0.smithi102.stdout:56: delete oid 12 current snap is 1 2024-09-17T17:29:48.670 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:48.670 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:48.673 INFO:tasks.rados.rados.0.smithi102.stdout:55: expect (ObjNum 17 snap 0 seq_num 17) 2024-09-17T17:29:48.673 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.677 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 239402 2024-09-17T17:29:48.677 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:29:48.684 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:48.684 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:48.704 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:48.710 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:48.707+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:48.726 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:48.726 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:48.747 INFO:tasks.rados.rados.0.smithi102.stdout:50: finishing write tid 3 to smithi10238019-50 2024-09-17T17:29:48.748 INFO:tasks.rados.rados.0.smithi102.stdout:53: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-17T17:29:48.748 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.753 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.753 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.754 INFO:tasks.rados.rados.0.smithi102.stdout:50: finishing write tid 4 to smithi10238019-50 2024-09-17T17:29:48.754 INFO:tasks.rados.rados.0.smithi102.stdout:50: oid 50 updating version 0 to 17 2024-09-17T17:29:48.755 INFO:tasks.rados.rados.0.smithi102.stdout:50: oid 50 version 17 is already newer than 16 2024-09-17T17:29:48.755 INFO:tasks.rados.rados.0.smithi102.stdout:50: oid 50 updating version 17 to 18 2024-09-17T17:29:48.755 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 50 v 18 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-17T17:29:48.755 INFO:tasks.rados.rados.0.smithi102.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-17T17:29:48.795 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:48.795 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:48.795 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:29:48.795 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:48.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:29:48.845 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:48.848 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:48.846+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:48.867 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:48.887 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:48.887 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:48.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:48.911 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:48.911 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:48.911 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:29:48.911 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:29:48.911 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:48.912 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:48.910+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:48.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:29:48.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:48.930 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:48.952 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:48.972 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:48.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:48.989 INFO:tasks.rados.rados.0.smithi102.stdout:49: finishing write tid 3 to smithi10238019-49 2024-09-17T17:29:48.990 INFO:tasks.rados.rados.0.smithi102.stdout:49: finishing write tid 4 to smithi10238019-49 2024-09-17T17:29:48.990 INFO:tasks.rados.rados.0.smithi102.stdout:49: oid 49 updating version 0 to 9 2024-09-17T17:29:48.990 INFO:tasks.rados.rados.0.smithi102.stdout:49: oid 49 version 9 is already newer than 7 2024-09-17T17:29:48.990 INFO:tasks.rados.rados.0.smithi102.stdout:49: oid 49 version 9 is already newer than 8 2024-09-17T17:29:48.990 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-17T17:29:48.990 INFO:tasks.rados.rados.0.smithi102.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-17T17:29:48.991 INFO:tasks.rados.rados.0.smithi102.stdout:49: done (4 left) 2024-09-17T17:29:48.991 INFO:tasks.rados.rados.0.smithi102.stdout:50: done (3 left) 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:53: done (2 left) 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:55: done (1 left) 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:56: done (0 left) 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:57: rollback oid 42 current snap is 1 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 42 to 0 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:58: rmattr oid 33 current snap is 1 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:58: done (1 left) 2024-09-17T17:29:48.992 INFO:tasks.rados.rados.0.smithi102.stdout:59: read oid 6 snap -1 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:59: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:60: read oid 1 snap -1 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:60: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:61: read oid 35 snap 0 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:61: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:62: rollback oid 39 current snap is 1 2024-09-17T17:29:48.993 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 39 to 0 2024-09-17T17:29:48.994 INFO:tasks.rados.rados.0.smithi102.stdout:63: delete oid 44 current snap is 1 2024-09-17T17:29:48.994 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:48.994 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:60: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 25342 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:60: done (5 left) 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:63: done (4 left) 2024-09-17T17:29:48.996 INFO:tasks.rados.rados.0.smithi102.stdout:64: snap_create 2024-09-17T17:29:48.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:29:48.998+0000 7fc25ffff640 0 -- 172.21.15.7:0/137490859 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fc268221050 msgr2=0x7fc268223440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:29:49.001 INFO:tasks.rados.rados.0.smithi102.stdout:57: finishing rollback tid 0 to smithi10238019-42 2024-09-17T17:29:49.001 INFO:tasks.rados.rados.0.smithi102.stdout:57: finishing rollback tid 1 to smithi10238019-42 2024-09-17T17:29:49.004 INFO:tasks.rados.rados.0.smithi102.stdout:62: finishing rollback tid 0 to smithi10238019-39 2024-09-17T17:29:49.005 INFO:tasks.rados.rados.0.smithi102.stdout:62: finishing rollback tid 1 to smithi10238019-39 2024-09-17T17:29:49.005 INFO:tasks.rados.rados.0.smithi102.stdout:62: finishing rollback tid 2 to smithi10238019-39 2024-09-17T17:29:49.005 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 39 v 8 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-17T17:29:49.012 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:49.010+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:49.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:29:49.043 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:49.070 INFO:tasks.rados.rados.0.smithi102.stdout:57: finishing rollback tid 2 to smithi10238019-42 2024-09-17T17:29:49.070 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 42 v 13 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-17T17:29:49.073 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.073 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:29:49.073 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:29:49.073 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.59426732699999996, 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:29:49.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.076 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.077 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:29:49.077 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:29:49.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.077 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.6s4 37/35 ECSubWrite(tid=42, reqid=client.4566.0:152, at_version=0'0, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.746300+0000", 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.32475125199999999, 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072353700000000001, 2024-09-17T17:29:49.079 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746300+0000", 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746300+0000", 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746300+0000", 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5099999999999999e-07 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746310+0000", 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0259e-05 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746333+0000", 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3320999999999999e-05 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746344+0000", 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0658e-05 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746427+0000", 2024-09-17T17:29:49.082 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2631000000000002e-05 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746443+0000", 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5875e-05 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.746995+0000", 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00055247399999999996 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.747023+0000", 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8167999999999999e-05 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/35 ECSubWrite(tid=13, reqid=client.4566.0:148, at_version=0'0, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.988005+0000", 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.083046364999999997, 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054578999999999999, 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988005+0000", 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988005+0000", 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.085 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988006+0000", 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0020000000000001e-06 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988020+0000", 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4626e-05 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988045+0000", 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4655e-05 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988052+0000", 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7039999999999997e-06 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988100+0000", 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8712999999999999e-05 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.087 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988105+0000", 2024-09-17T17:29:49.088 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.096e-06 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988524+0000", 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00041827800000000001 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988550+0000", 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6716e-05 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.089 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:179 3.4s0 3:26d49efd:::smithi10238019-44:head [delete] snapc 2=[2] ondisk+write+known_if_redirected+full_force+supports_pool_eio e37)", 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.080872713999999998, 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039941960000000002, 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 179 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.091 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990179+0000", 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.22e-06 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990184+0000", 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1239999999999998e-06 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990194+0000", 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0377000000000001e-05 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990200+0000", 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5690000000000001e-06 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990248+0000", 2024-09-17T17:29:49.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7615999999999999e-05 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990300+0000", 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2364000000000002e-05 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991074+0000", 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077367199999999999 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992851+0000", 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001776632 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994099+0000", 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012486610000000001 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994172+0000", 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2960999999999995e-05 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991848+0000", 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.079202986000000003, 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015108260000000001, 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.096 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991848+0000", 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991848+0000", 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991849+0000", 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3799999999999996e-07 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991867+0000", 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8641999999999999e-05 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991897+0000", 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9747000000000002e-05 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991905+0000", 2024-09-17T17:29:49.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2290000000000003e-06 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991966+0000", 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1191000000000004e-05 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991975+0000", 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4090000000000003e-06 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993335+0000", 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001360106 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993359+0000", 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3764000000000002e-05 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=8, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991996+0000", 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.079054757000000003, 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000110537, 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.101 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991996+0000", 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991996+0000", 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.103 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.103 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991997+0000", 2024-09-17T17:29:49.103 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:29:49.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.103 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.104 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992002+0000", 2024-09-17T17:29:49.104 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0139999999999998e-06 2024-09-17T17:29:49.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.104 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.104 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992030+0000", 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7716e-05 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992037+0000", 2024-09-17T17:29:49.105 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2239999999999998e-06 2024-09-17T17:29:49.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.106 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.106 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.106 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992088+0000", 2024-09-17T17:29:49.106 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1254000000000001e-05 2024-09-17T17:29:49.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992107+0000", 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8512000000000002e-05 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.107 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=8, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.108 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.992440+0000", 2024-09-17T17:29:49.108 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.078610942000000003, 2024-09-17T17:29:49.108 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00041198899999999998, 2024-09-17T17:29:49.108 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992440+0000", 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.109 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992440+0000", 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992441+0000", 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4600000000000004e-07 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992445+0000", 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9439999999999998e-06 2024-09-17T17:29:49.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992510+0000", 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5627e-05 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.111 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992517+0000", 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3019999999999997e-06 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992831+0000", 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00031377 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992852+0000", 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0699999999999998e-05 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.112 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/35 ECSubWrite(tid=16, reqid=client.4566.0:165, at_version=37'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.992456+0000", 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.078594521000000001, 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013329380000000001, 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.113 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992456+0000", 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992456+0000", 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992457+0000", 2024-09-17T17:29:49.114 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0499999999999999e-07 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992475+0000", 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7822999999999999e-05 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993009+0000", 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00053462200000000001 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.115 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993017+0000", 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4490000000000002e-06 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993065+0000", 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8316000000000003e-05 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993072+0000", 2024-09-17T17:29:49.116 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9e-06 2024-09-17T17:29:49.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.117 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.117 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993753+0000", 2024-09-17T17:29:49.117 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068165200000000004 2024-09-17T17:29:49.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993789+0000", 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5871000000000003e-05 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=8, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.992726+0000", 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.078325149999999996, 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000167842, 2024-09-17T17:29:49.118 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992726+0000", 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.119 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992726+0000", 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992726+0000", 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2099999999999996e-07 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992731+0000", 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.481e-06 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992759+0000", 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7596000000000001e-05 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992765+0000", 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7660000000000001e-06 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.121 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992876+0000", 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000110479 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992894+0000", 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7799e-05 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.122 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=8, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.992871+0000", 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.078180030999999997, 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107738, 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992871+0000", 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992871+0000", 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992871+0000", 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7899999999999998e-07 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992875+0000", 2024-09-17T17:29:49.124 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8709999999999999e-06 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992898+0000", 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2830999999999999e-05 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992905+0000", 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0310000000000001e-06 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.125 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992964+0000", 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8359999999999997e-05 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992979+0000", 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5065999999999999e-05 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=8, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.993242+0000", 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.077809183000000004, 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109279, 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993242+0000", 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993242+0000", 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993242+0000", 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.128 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993247+0000", 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5410000000000002e-06 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993280+0000", 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2595000000000002e-05 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.130 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.130 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993287+0000", 2024-09-17T17:29:49.130 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3969999999999997e-06 2024-09-17T17:29:49.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.130 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.131 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993334+0000", 2024-09-17T17:29:49.131 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7012999999999999e-05 2024-09-17T17:29:49.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.131 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.131 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993351+0000", 2024-09-17T17:29:49.132 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7005000000000001e-05 2024-09-17T17:29:49.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.132 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.133 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/35 ECSubWrite(tid=17, reqid=client.4566.0:166, at_version=37'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.133 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.993398+0000", 2024-09-17T17:29:49.133 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.077653004999999997, 2024-09-17T17:29:49.133 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067937, 2024-09-17T17:29:49.133 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993398+0000", 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993398+0000", 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.135 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993399+0000", 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8699999999999995e-07 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993414+0000", 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5260999999999998e-05 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993445+0000", 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1358e-05 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993453+0000", 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0690000000000002e-06 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993525+0000", 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1324999999999999e-05 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993535+0000", 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0353000000000001e-05 2024-09-17T17:29:49.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994046+0000", 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00051097899999999995 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994077+0000", 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1438e-05 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=8, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.993903+0000", 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.077147733999999996, 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00065609200000000005, 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.139 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993903+0000", 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.140 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993903+0000", 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993904+0000", 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0900000000000001e-07 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993907+0000", 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2789999999999999e-06 2024-09-17T17:29:49.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993948+0000", 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0992000000000003e-05 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993959+0000", 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1398e-05 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.142 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994019+0000", 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9058000000000003e-05 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994559+0000", 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054075599999999998 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.143 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994892+0000", 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.076159291000000004, 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016331200000000001, 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994892+0000", 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994892+0000", 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.145 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994892+0000", 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6499999999999998e-07 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994900+0000", 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0269999999999995e-06 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.146 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994961+0000", 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0677999999999997e-05 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994970+0000", 2024-09-17T17:29:49.147 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8799999999999997e-06 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995039+0000", 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8522000000000006e-05 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6439999999999998e-05 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.076001088999999994, 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.149 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:29:49.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.151 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:29:49.152 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:29:49.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.152 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.152 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:29:49.152 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.153 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995151+0000", 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.075900308999999999, 2024-09-17T17:29:49.154 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000119116, 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995151+0000", 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995151+0000", 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995151+0000", 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0699999999999996e-07 2024-09-17T17:29:49.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995159+0000", 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4390000000000003e-06 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995191+0000", 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1758000000000003e-05 2024-09-17T17:29:49.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995197+0000", 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9650000000000002e-06 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995254+0000", 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6155e-05 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995270+0000", 2024-09-17T17:29:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6092000000000001e-05 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995218+0000", 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.075832994000000001, 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8083000000000004e-05, 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995218+0000", 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.160 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.161 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995218+0000", 2024-09-17T17:29:49.161 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.161 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.161 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995219+0000", 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6499999999999999e-07 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995223+0000", 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5170000000000001e-06 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995253+0000", 2024-09-17T17:29:49.162 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9862e-05 2024-09-17T17:29:49.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.163 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.163 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.163 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995259+0000", 2024-09-17T17:29:49.163 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5039999999999999e-06 2024-09-17T17:29:49.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.164 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.164 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.164 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995289+0000", 2024-09-17T17:29:49.164 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9082999999999999e-05 2024-09-17T17:29:49.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995296+0000", 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.452e-06 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.165 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995640+0000", 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.075411354999999999, 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000100226, 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995640+0000", 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.167 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.167 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.167 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995640+0000", 2024-09-17T17:29:49.167 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995640+0000", 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.37e-07 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.168 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995644+0000", 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6339999999999999e-06 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995671+0000", 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7696999999999999e-05 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.169 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995679+0000", 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1280000000000002e-06 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995722+0000", 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3291999999999998e-05 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.170 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995740+0000", 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7938e-05 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:49.171 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995661+0000", 2024-09-17T17:29:49.172 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.075390086999999995, 2024-09-17T17:29:49.172 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012634000000000001, 2024-09-17T17:29:49.172 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.172 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995661+0000", 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995661+0000", 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995664+0000", 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8609999999999998e-06 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995670+0000", 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8880000000000002e-06 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.174 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995682+0000", 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2748000000000001e-05 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995699+0000", 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7082e-05 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995767+0000", 2024-09-17T17:29:49.175 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7883e-05 2024-09-17T17:29:49.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.176 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.176 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.176 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995787+0000", 2024-09-17T17:29:49.176 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9877999999999998e-05 2024-09-17T17:29:49.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.177 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.177 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.177 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:49.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:29:49.180 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:49.177+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:49.280 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:49.278+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:49.291 INFO:tasks.rados.rados.0.smithi102.stdout:61: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-17T17:29:49.293 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 297244 2024-09-17T17:29:49.298 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 501714 2024-09-17T17:29:49.300 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 194830 2024-09-17T17:29:49.327 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:49.327 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:29:49.327 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:49.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:29:49.380 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:49.378+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:49.443 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:29:49.443 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:29:49.444 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:49.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:29:49.481 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:49.479+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:49.582 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:49.580+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=13, reqid=client.4566.0:148, at_version=0'0, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.988106+0000", 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.61133219000000005, 2024-09-17T17:29:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058845800000000004, 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988106+0000", 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988106+0000", 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988106+0000", 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6e-07 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988114+0000", 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4220000000000003e-06 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988150+0000", 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6230999999999999e-05 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988160+0000", 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5310000000000007e-06 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988226+0000", 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6357000000000002e-05 2024-09-17T17:29:49.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988236+0000", 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0188e-05 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988668+0000", 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000431674 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.605 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.988695+0000", 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6894999999999999e-05 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.cs2 37/32 ECSubRead(tid=26, to_read={3:30756a83:::smithi10238019-35:head=73728,69632,0}, subchunks={3:30756a83:::smithi10238019-35:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.990616+0000", 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.60882199100000001, 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00042859099999999998, 2024-09-17T17:29:49.606 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990616+0000", 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990616+0000", 2024-09-17T17:29:49.607 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990617+0000", 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.77e-07 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990623+0000", 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3199999999999996e-06 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.608 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990678+0000", 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4984000000000001e-05 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990688+0000", 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8679999999999994e-06 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990776+0000", 2024-09-17T17:29:49.609 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8058000000000003e-05 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991045+0000", 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000268984 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:49.610 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.60767278800000002, 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.611 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:29:49.612 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.613 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:29:49.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=16, reqid=client.4566.0:165, at_version=37'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.60538049999999999, 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0014151649999999999, 2024-09-17T17:29:49.615 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:49.616 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994059+0000", 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7000000000000004e-07 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994076+0000", 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7873999999999999e-05 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994174+0000", 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7345999999999999e-05 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994183+0000", 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7739999999999993e-06 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994549+0000", 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00036644599999999998 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994559+0000", 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0019e-05 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995447+0000", 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00088779500000000003 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995473+0000", 2024-09-17T17:29:49.619 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6241000000000002e-05 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=17, reqid=client.4566.0:166, at_version=37'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.60535848800000003, 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001407352, 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.621 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6699999999999999e-07 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994094+0000", 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3264e-05 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994201+0000", 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107019 2024-09-17T17:29:49.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994206+0000", 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4519999999999996e-06 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994836+0000", 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000629709 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.623 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994844+0000", 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.841e-06 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995476+0000", 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00063211799999999998 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995487+0000", 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1582e-05 2024-09-17T17:29:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.cs2 37/32 ECSubRead(tid=27, to_read={3:30756a83:::smithi10238019-35:head=122880,20480,0}, subchunks={3:30756a83:::smithi10238019-35:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:49.001354+0000", 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.598084167, 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000162683, 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.625 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001354+0000", 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001354+0000", 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.626 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001355+0000", 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.85e-07 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001363+0000", 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5859999999999999e-06 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001401+0000", 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8708000000000003e-05 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001408+0000", 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2890000000000003e-06 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001453+0000", 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5173000000000003e-05 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.628 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001517+0000", 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4141999999999997e-05 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.cs2 37/32 ECSubRead(tid=28, to_read={3:30756a83:::smithi10238019-35:head=45056,98304,0}, subchunks={3:30756a83:::smithi10238019-35:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:49.001385+0000", 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.59805304599999998, 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00028426299999999999, 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001385+0000", 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001385+0000", 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001386+0000", 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8400000000000001e-07 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001388+0000", 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8559999999999998e-06 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.631 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001427+0000", 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8130000000000003e-05 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001431+0000", 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7400000000000004e-06 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001536+0000", 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000105021 2024-09-17T17:29:49.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.001670+0000", 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000133332 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:49.069383+0000", 2024-09-17T17:29:49.633 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.530055838, 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00087504900000000003, 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069383+0000", 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.634 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069383+0000", 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069384+0000", 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1939999999999999e-06 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069397+0000", 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3295000000000001e-05 2024-09-17T17:29:49.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069431+0000", 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3989000000000003e-05 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069440+0000", 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7350000000000002e-06 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.636 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069508+0000", 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8403000000000002e-05 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069522+0000", 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.343e-05 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.070224+0000", 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00070213100000000004 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.070258+0000", 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3871999999999997e-05 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.638 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:49.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:29:49.665 INFO:tasks.rados.rados.0.smithi102.stdout:59: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-17T17:29:49.665 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:49.666 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:49.668 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 83682 2024-09-17T17:29:49.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:29:49.671 INFO:tasks.rados.rados.0.smithi102.stdout:57: done (4 left) 2024-09-17T17:29:49.672 INFO:tasks.rados.rados.0.smithi102.stdout:59: done (3 left) 2024-09-17T17:29:49.672 INFO:tasks.rados.rados.0.smithi102.stdout:61: done (2 left) 2024-09-17T17:29:49.672 INFO:tasks.rados.rados.0.smithi102.stdout:62: done (1 left) 2024-09-17T17:29:49.672 INFO:tasks.rados.rados.0.smithi102.stdout:64: done (0 left) 2024-09-17T17:29:49.672 INFO:tasks.rados.rados.0.smithi102.stdout:65: snap_remove snap 0 2024-09-17T17:29:49.682 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:49.681+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:49.747 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:49.747 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:29:49.747 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:49.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:29:49.783 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:49.781+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:49.863 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:29:49.863 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:29:49.864 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:49.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:29:49.883 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:49.881+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:49.979 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_repop(client.4311.0:5 2.2 e27/25 2:5cea7035:::rbd_info:head v 27'1, mlcod=0'0)", 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:32.596512+0000", 2024-09-17T17:29:49.980 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.381550498999999, 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011037219999999999, 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.981 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596512+0000", 2024-09-17T17:29:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596512+0000", 2024-09-17T17:29:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.983 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596512+0000", 2024-09-17T17:29:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8599999999999999e-07 2024-09-17T17:29:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.984 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596518+0000", 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7799999999999997e-06 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596557+0000", 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9292000000000003e-05 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596568+0000", 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0467e-05 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596650+0000", 2024-09-17T17:29:49.986 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2490999999999993e-05 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.596676+0000", 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5729e-05 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.597566+0000", 2024-09-17T17:29:49.987 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000890664 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:32.597615+0000", 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9013e-05 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:49.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_repop(client.4311.0:7 2.2 e28/25 2:5cea7035:::rbd_info:head v 28'2, mlcod=0'0)", 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:33.593015+0000", 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.385047449999998, 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072452200000000001, 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:49.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593015+0000", 2024-09-17T17:29:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593015+0000", 2024-09-17T17:29:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.992 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593015+0000", 2024-09-17T17:29:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3900000000000001e-07 2024-09-17T17:29:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:49.993 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593022+0000", 2024-09-17T17:29:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9009999999999997e-06 2024-09-17T17:29:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:49.994 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593061+0000", 2024-09-17T17:29:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8924999999999997e-05 2024-09-17T17:29:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593076+0000", 2024-09-17T17:29:49.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5410999999999999e-05 2024-09-17T17:29:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593128+0000", 2024-09-17T17:29:49.996 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2209000000000001e-05 2024-09-17T17:29:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:49.997 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593144+0000", 2024-09-17T17:29:49.998 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5384e-05 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593702+0000", 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00055858899999999996 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:49.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:33.593739+0000", 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6863999999999998e-05 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s2 37/35 ECSubWrite(tid=8, reqid=client.4566.0:179, at_version=37'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.98702426300000001, 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021580079999999999, 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:50.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6e-07 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991063+0000", 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4984999999999999e-05 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:50.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991151+0000", 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7474999999999999e-05 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991160+0000", 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2569999999999996e-06 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991232+0000", 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1891999999999995e-05 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991243+0000", 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1243999999999999e-05 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993161+0000", 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019180689999999999 2024-09-17T17:29:50.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993196+0000", 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4826000000000002e-05 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s2 37/35 ECSubWrite(tid=9, reqid=client.4566.0:179, at_version=0'0, trim_to=0'0, roll_forward_to=37'6))", 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994347+0000", 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.98371556900000001, 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00083593999999999997, 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:50.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994347+0000", 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994347+0000", 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994347+0000", 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2200000000000003e-07 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994358+0000", 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0776e-05 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994409+0000", 2024-09-17T17:29:50.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1140000000000002e-05 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994420+0000", 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1043000000000001e-05 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994473+0000", 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2868999999999998e-05 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994482+0000", 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2220000000000008e-06 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995145+0000", 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00066284799999999995 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995183+0000", 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7320000000000002e-05 2024-09-17T17:29:50.010 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.011 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.011 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:50.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:29:50.013 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:50.010+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:50.113 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:50.111+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:50.120 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:50.120 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:29:50.120 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:50.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:29:50.214 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:50.212+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:50.275 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:50.275 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:29:50.276 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:29:50.276 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:50.284 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:29:50.329 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:50.327+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s1 37/35 ECSubWrite(tid=17, reqid=client.4566.0:166, at_version=37'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.993427+0000", 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.3971146939999999, 2024-09-17T17:29:50.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019963619999999998, 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993427+0000", 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:50.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993427+0000", 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993427+0000", 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.99e-07 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993443+0000", 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5585000000000001e-05 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993520+0000", 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7095000000000006e-05 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993528+0000", 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7160000000000003e-06 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000530022 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994067+0000", 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1910000000000006e-06 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995400+0000", 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013329259999999999 2024-09-17T17:29:50.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995423+0000", 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3127999999999999e-05 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s6 37/35 ECSubWrite(tid=9, reqid=client.4566.0:179, at_version=0'0, trim_to=0'0, roll_forward_to=37'6))", 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994501+0000", 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.3960401739999999, 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011298479999999999, 2024-09-17T17:29:50.397 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994501+0000", 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994501+0000", 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994502+0000", 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3499999999999995e-07 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994511+0000", 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4010000000000003e-06 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994561+0000", 2024-09-17T17:29:50.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0034999999999998e-05 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994570+0000", 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5010000000000004e-06 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994629+0000", 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9725999999999997e-05 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994638+0000", 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3000000000000002e-06 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995593+0000", 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095504100000000001 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995631+0000", 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8108999999999997e-05 2024-09-17T17:29:50.401 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.cs1 37/32 ECSubRead(tid=28, to_read={3:30756a83:::smithi10238019-35:head=45056,98304,0}, subchunks={3:30756a83:::smithi10238019-35:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.999089+0000", 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.3914526519999999, 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00024424499999999998, 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:50.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999089+0000", 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999089+0000", 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999089+0000", 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6700000000000003e-07 2024-09-17T17:29:50.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999098+0000", 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4149999999999999e-06 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999123+0000", 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5255000000000001e-05 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999131+0000", 2024-09-17T17:29:50.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6009999999999997e-06 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999185+0000", 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4707000000000003e-05 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.999333+0000", 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014760000000000001 2024-09-17T17:29:50.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s1 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:49.069399+0000", 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.321141772, 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00084692299999999997, 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.406 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069399+0000", 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069399+0000", 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069400+0000", 2024-09-17T17:29:50.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9899999999999999e-07 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069410+0000", 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2269999999999999e-06 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069437+0000", 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7039e-05 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:50.408 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069446+0000", 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4620000000000002e-06 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069522+0000", 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6279999999999995e-05 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.069532+0000", 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1989999999999995e-06 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.070210+0000", 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067870799999999998 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.070246+0000", 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6109000000000003e-05 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:50.411 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:50.411 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:50.412 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:29:50.453 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:50.451+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:50.517 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:50.515+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:29:50.532 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:50.532 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:50.532 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:29:50.533 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:50.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:29:50.617 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:50.615+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:50.649 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:50.649 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:50.649 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:29:50.649 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:29:50.650 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:50.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:29:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:29:50.679 INFO:tasks.rados.rados.0.smithi102.stdout:65: done (0 left) 2024-09-17T17:29:50.679 INFO:tasks.rados.rados.0.smithi102.stdout:66: copy_from oid 22 from oid 36 current snap is 2 2024-09-17T17:29:50.679 INFO:tasks.rados.rados.0.smithi102.stdout:67: rollback oid 3 current snap is 2 2024-09-17T17:29:50.680 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 3 to 1 2024-09-17T17:29:50.680 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 1 current snap is 2 2024-09-17T17:29:50.680 INFO:tasks.rados.rados.0.smithi102.stdout:68: seq_num 52 ranges {32768=507904,540672=376832} 2024-09-17T17:29:50.682 INFO:tasks.rados.rados.0.smithi102.stdout:68: writing smithi10238019-1 from 32768 to 540672 tid 1 2024-09-17T17:29:50.684 INFO:tasks.rados.rados.0.smithi102.stdout:68: writing smithi10238019-1 from 540672 to 917504 tid 2 2024-09-17T17:29:50.684 INFO:tasks.rados.rados.0.smithi102.stdout:69: setattr oid 32 current snap is 2 2024-09-17T17:29:50.685 INFO:tasks.rados.rados.0.smithi102.stdout:67: finishing rollback tid 0 to smithi10238019-3 2024-09-17T17:29:50.685 INFO:tasks.rados.rados.0.smithi102.stdout:67: finishing rollback tid 1 to smithi10238019-3 2024-09-17T17:29:50.685 INFO:tasks.rados.rados.0.smithi102.stdout:67: finishing rollback tid 2 to smithi10238019-3 2024-09-17T17:29:50.686 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 3 v 12 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-17T17:29:50.687 INFO:tasks.rados.rados.0.smithi102.stdout:67: done (3 left) 2024-09-17T17:29:50.687 INFO:tasks.rados.rados.0.smithi102.stdout:70: read oid 11 snap -1 2024-09-17T17:29:50.687 INFO:tasks.rados.rados.0.smithi102.stdout:70: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-17T17:29:50.687 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 4 current snap is 2 2024-09-17T17:29:50.687 INFO:tasks.rados.rados.0.smithi102.stdout:71: seq_num 53 ranges {65536=524288,589824=376832} 2024-09-17T17:29:50.690 INFO:tasks.rados.rados.0.smithi102.stdout:71: writing smithi10238019-4 from 65536 to 589824 tid 1 2024-09-17T17:29:50.693 INFO:tasks.rados.rados.0.smithi102.stdout:71: writing smithi10238019-4 from 589824 to 966656 tid 2 2024-09-17T17:29:50.693 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 23 current snap is 2 2024-09-17T17:29:50.693 INFO:tasks.rados.rados.0.smithi102.stdout:72: seq_num 54 ranges {376832=540672,917504=376832} 2024-09-17T17:29:50.698 INFO:tasks.rados.rados.0.smithi102.stdout:72: writing smithi10238019-23 from 376832 to 917504 tid 1 2024-09-17T17:29:50.701 INFO:tasks.rados.rados.0.smithi102.stdout:72: writing smithi10238019-23 from 917504 to 1294336 tid 2 2024-09-17T17:29:50.701 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 44 current snap is 2 2024-09-17T17:29:50.701 INFO:tasks.rados.rados.0.smithi102.stdout:68: finishing write tid 1 to smithi10238019-1 2024-09-17T17:29:50.701 INFO:tasks.rados.rados.0.smithi102.stdout:73: seq_num 55 ranges {0=557056,557056=376832} 2024-09-17T17:29:50.704 INFO:tasks.rados.rados.0.smithi102.stdout:73: writing smithi10238019-44 from 0 to 557056 tid 1 2024-09-17T17:29:50.707 INFO:tasks.rados.rados.0.smithi102.stdout:73: writing smithi10238019-44 from 557056 to 933888 tid 2 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:74: copy_from oid 34 from oid 13 current snap is 2 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: finishing write tid 2 to smithi10238019-1 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: finishing write tid 3 to smithi10238019-1 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: finishing write tid 4 to smithi10238019-1 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: oid 1 updating version 0 to 19 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: oid 1 updating version 19 to 21 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: oid 1 version 21 is already newer than 20 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 1 v 21 (ObjNum 52 snap 2 seq_num 52) dirty exists 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:68: left oid 1 (ObjNum 52 snap 2 seq_num 52) 2024-09-17T17:29:50.708 INFO:tasks.rados.rados.0.smithi102.stdout:71: finishing write tid 1 to smithi10238019-4 2024-09-17T17:29:50.709 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 32 v 17 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-17T17:29:50.709 INFO:tasks.rados.rados.0.smithi102.stdout:70: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-17T17:29:50.709 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:50.709 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 114647 2024-09-17T17:29:50.712 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 168153 2024-09-17T17:29:50.712 INFO:tasks.rados.rados.0.smithi102.stdout:71: finishing write tid 2 to smithi10238019-4 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:71: finishing write tid 3 to smithi10238019-4 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:68: done (7 left) 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:69: done (6 left) 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:70: done (5 left) 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:75: rollback oid 46 current snap is 2 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 46 to 1 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:71: finishing write tid 4 to smithi10238019-4 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:71: oid 4 updating version 0 to 6 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:71: oid 4 updating version 6 to 8 2024-09-17T17:29:50.713 INFO:tasks.rados.rados.0.smithi102.stdout:71: oid 4 version 8 is already newer than 7 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 4 v 8 (ObjNum 53 snap 2 seq_num 53) dirty exists 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:71: left oid 4 (ObjNum 53 snap 2 seq_num 53) 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:66: finishing copy_from racing read to smithi10238019-22 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: finishing write tid 1 to smithi10238019-23 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: finishing write tid 2 to smithi10238019-23 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: finishing write tid 3 to smithi10238019-23 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: finishing write tid 4 to smithi10238019-23 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: oid 23 updating version 0 to 12 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: oid 23 version 12 is already newer than 11 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: oid 23 version 12 is already newer than 10 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 23 v 12 (ObjNum 54 snap 2 seq_num 54) dirty exists 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:72: left oid 23 (ObjNum 54 snap 2 seq_num 54) 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:66: finishing copy_from to smithi10238019-22 2024-09-17T17:29:50.714 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 22 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:66: done (5 left) 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:71: done (4 left) 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:72: done (3 left) 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:76: delete oid 14 current snap is 2 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:73: finishing write tid 1 to smithi10238019-44 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:76: done (3 left) 2024-09-17T17:29:50.715 INFO:tasks.rados.rados.0.smithi102.stdout:77: read oid 7 snap -1 2024-09-17T17:29:50.716 INFO:tasks.rados.rados.0.smithi102.stdout:77: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-17T17:29:50.716 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 27 current snap is 2 2024-09-17T17:29:50.716 INFO:tasks.rados.rados.0.smithi102.stdout:78: seq_num 57 ranges {442368=589824,1032192=376832} 2024-09-17T17:29:50.718 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:50.716+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:50.719 INFO:tasks.rados.rados.0.smithi102.stdout:78: writing smithi10238019-27 from 442368 to 1032192 tid 1 2024-09-17T17:29:50.722 INFO:tasks.rados.rados.0.smithi102.stdout:78: writing smithi10238019-27 from 1032192 to 1409024 tid 2 2024-09-17T17:29:50.722 INFO:tasks.rados.rados.0.smithi102.stdout:79: delete oid 32 current snap is 2 2024-09-17T17:29:50.722 INFO:tasks.rados.rados.0.smithi102.stdout:74: finishing copy_from racing read to smithi10238019-34 2024-09-17T17:29:50.722 INFO:tasks.rados.rados.0.smithi102.stdout:73: finishing write tid 2 to smithi10238019-44 2024-09-17T17:29:50.722 INFO:tasks.rados.rados.0.smithi102.stdout:77: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-17T17:29:50.722 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:50.724 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:50.725 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 51414 2024-09-17T17:29:50.728 INFO:tasks.rados.rados.0.smithi102.stdout:75: finishing rollback tid 0 to smithi10238019-46 2024-09-17T17:29:50.728 INFO:tasks.rados.rados.0.smithi102.stdout:75: finishing rollback tid 1 to smithi10238019-46 2024-09-17T17:29:50.728 INFO:tasks.rados.rados.0.smithi102.stdout:75: finishing rollback tid 2 to smithi10238019-46 2024-09-17T17:29:50.728 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-17T17:29:50.728 INFO:tasks.rados.rados.0.smithi102.stdout:74: finishing copy_from to smithi10238019-34 2024-09-17T17:29:50.728 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 34 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-17T17:29:50.729 INFO:tasks.rados.rados.0.smithi102.stdout:74: done (5 left) 2024-09-17T17:29:50.729 INFO:tasks.rados.rados.0.smithi102.stdout:75: done (4 left) 2024-09-17T17:29:50.729 INFO:tasks.rados.rados.0.smithi102.stdout:77: done (3 left) 2024-09-17T17:29:50.729 INFO:tasks.rados.rados.0.smithi102.stdout:79: done (2 left) 2024-09-17T17:29:50.729 INFO:tasks.rados.rados.0.smithi102.stdout:80: read oid 47 snap -1 2024-09-17T17:29:50.729 INFO:tasks.rados.rados.0.smithi102.stdout:80: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:81: delete oid 34 current snap is 2 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:73: finishing write tid 3 to smithi10238019-44 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:73: finishing write tid 4 to smithi10238019-44 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:73: oid 44 updating version 0 to 9 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:73: oid 44 updating version 9 to 11 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:73: oid 44 version 11 is already newer than 10 2024-09-17T17:29:50.730 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 44 v 11 (ObjNum 55 snap 2 seq_num 55) dirty exists 2024-09-17T17:29:50.731 INFO:tasks.rados.rados.0.smithi102.stdout:73: left oid 44 (ObjNum 55 snap 2 seq_num 55) 2024-09-17T17:29:50.731 INFO:tasks.rados.rados.0.smithi102.stdout:78: finishing write tid 1 to smithi10238019-27 2024-09-17T17:29:50.731 INFO:tasks.rados.rados.0.smithi102.stdout:78: finishing write tid 2 to smithi10238019-27 2024-09-17T17:29:50.732 INFO:tasks.rados.rados.0.smithi102.stdout:73: done (3 left) 2024-09-17T17:29:50.732 INFO:tasks.rados.rados.0.smithi102.stdout:81: done (2 left) 2024-09-17T17:29:50.732 INFO:tasks.rados.rados.0.smithi102.stdout:82: snap_remove snap 1 2024-09-17T17:29:50.779 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:50.779 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:29:50.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:29:50.779 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [] 2024-09-17T17:29:50.779 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:50.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:29:50.819 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:50.817+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:50.894 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:50.895 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:50.895 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:29:50.895 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:50.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:29:50.946 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:50.945+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:51.009 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:29:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:29:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:29:51.010 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:51.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:29:51.020 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:51.018+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:51.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:29:51.121 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:51.120+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:51.168 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:29:51.168 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:29:51.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:29:51.169 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:29:51.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.169 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.169 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:29:51.169 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.584347602000001, 2024-09-17T17:29:51.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.27275196200000001, 2024-09-17T17:29:51.170 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.171 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1 2024-09-17T17:29:51.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.172 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:29:51.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:29:51.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580026+0000", 2024-09-17T17:29:51.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.359e-06 2024-09-17T17:29:51.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580034+0000", 2024-09-17T17:29:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-17T17:29:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580048+0000", 2024-09-17T17:29:51.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3974e-05 2024-09-17T17:29:51.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580058+0000", 2024-09-17T17:29:51.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0064999999999999e-05 2024-09-17T17:29:51.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580111+0000", 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3248000000000002e-05 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580119+0000", 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1270000000000003e-06 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847220+0000", 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26710079599999997 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847357+0000", 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013739500000000001 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847505+0000", 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147188 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.850265+0000", 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002760396 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852701+0000", 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024356059999999999 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852744+0000", 2024-09-17T17:29:51.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3095e-05 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852777+0000", 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2784000000000003e-05 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.986961+0000", 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.177410771000002, 2024-09-17T17:29:51.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067482250000000001, 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 41 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.986961+0000", 2024-09-17T17:29:51.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.986961+0000", 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.986962+0000", 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5099999999999999e-07 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987036+0000", 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4197999999999997e-05 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987044+0000", 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3719999999999999e-06 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987048+0000", 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5680000000000001e-06 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987890+0000", 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084162500000000003 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987952+0000", 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1723999999999994e-05 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.988048+0000", 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6335999999999994e-05 2024-09-17T17:29:51.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.989712+0000", 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016640489999999999 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993629+0000", 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039173319999999999 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993665+0000", 2024-09-17T17:29:51.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5840000000000002e-05 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993710+0000", 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4530000000000002e-05 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176976623000002, 2024-09-17T17:29:51.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010915796, 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 46 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5400000000000002e-07 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987400+0000", 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1450000000000001e-06 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987405+0000", 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2739999999999998e-06 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987409+0000", 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1370000000000004e-06 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990238+0000", 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028288110000000001 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990293+0000", 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4657999999999997e-05 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990394+0000", 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010156 2024-09-17T17:29:51.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991880+0000", 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014857169999999999 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998207+0000", 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0063267899999999997 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998262+0000", 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998311+0000", 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9777999999999997e-05 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:51.193 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176943508000001, 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010963481000000001, 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 47 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5400000000000001e-07 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987478+0000", 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9118e-05 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987490+0000", 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1425e-05 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987502+0000", 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2354999999999999e-05 2024-09-17T17:29:51.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990908+0000", 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003406002 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990958+0000", 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9597e-05 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991063+0000", 2024-09-17T17:29:51.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010533 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993063+0000", 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019997909999999999 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998320+0000", 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052566959999999999 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998349+0000", 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9200000000000002e-05 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998392+0000", 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3412999999999999e-05 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:51.199 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176859314000001, 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010965536999999999, 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 48 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5899999999999996e-07 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987518+0000", 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0429999999999998e-06 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987523+0000", 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3499999999999999e-06 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987526+0000", 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5159999999999999e-06 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991339+0000", 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038128670000000002 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991396+0000", 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7219000000000002e-05 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991498+0000", 2024-09-17T17:29:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010142 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993114+0000", 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016166900000000001 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998400+0000", 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052850600000000003 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998438+0000", 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8272000000000002e-05 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998478+0000", 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0540999999999997e-05 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:51.205 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176840964, 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011028196000000001, 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 49 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987534+0000", 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4399999999999999e-06 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987538+0000", 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9369999999999997e-06 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987541+0000", 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0350000000000002e-06 2024-09-17T17:29:51.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991920+0000", 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043794000000000003 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991972+0000", 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1412000000000002e-05 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992057+0000", 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4917e-05 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993157+0000", 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011003810000000001 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998485+0000", 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053282989999999999 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998517+0000", 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2019e-05 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998559+0000", 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2076000000000002e-05 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.211 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:51.212 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176819474999999, 2024-09-17T17:29:51.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011080325, 2024-09-17T17:29:51.212 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 50 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-07 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987605+0000", 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2114000000000003e-05 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987631+0000", 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5766999999999999e-05 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987646+0000", 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515999999999998e-05 2024-09-17T17:29:51.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992371+0000", 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047247360000000002 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992433+0000", 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1451000000000005e-05 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992522+0000", 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9621000000000001e-05 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994510+0000", 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001988227 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998564+0000", 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040533710000000001 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998595+0000", 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0843000000000003e-05 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998633+0000", 2024-09-17T17:29:51.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8321000000000001e-05 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176707449999999, 2024-09-17T17:29:51.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011043702000000001, 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 51 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:51.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987669+0000", 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7189999999999999e-06 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987674+0000", 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1780000000000002e-06 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987678+0000", 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5999999999999998e-06 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993192+0000", 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055137620000000002 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993262+0000", 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993349+0000", 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7362999999999997e-05 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994549+0000", 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011994110000000001 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998640+0000", 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040912409999999998 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998669+0000", 2024-09-17T17:29:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9209000000000001e-05 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998708+0000", 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9419000000000003e-05 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:51.225 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.176687359999999, 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011098023, 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 52 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:51.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2000000000000001e-07 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987688+0000", 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4729999999999999e-06 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987693+0000", 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5010000000000004e-06 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987698+0000", 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1499999999999998e-06 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994130+0000", 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064314890000000003 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994185+0000", 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5769000000000001e-05 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994272+0000", 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6638000000000006e-05 2024-09-17T17:29:51.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994990+0000", 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000717463 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998715+0000", 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003725 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998743+0000", 2024-09-17T17:29:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8830999999999999e-05 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998783+0000", 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9388999999999998e-05 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:58 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [cmpxattr striper.excl (26) op 1 mode 1 in=12b,setxattr striper.excl in=12b,call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.005578+0000", 2024-09-17T17:29:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.15879378, 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0082001399999999999, 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 58 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005578+0000", 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005578+0000", 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005579+0000", 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.051e-06 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005584+0000", 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9350000000000002e-06 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005595+0000", 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1060000000000001e-05 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005601+0000", 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3140000000000003e-06 2024-09-17T17:29:51.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005647+0000", 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6548999999999998e-05 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005704+0000", 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6317000000000003e-05 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.005837+0000", 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013316399999999999 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.006293+0000", 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00045653000000000002 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.013713+0000", 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074199009999999996 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.013750+0000", 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6857000000000003e-05 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.013779+0000", 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8462e-05 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.237 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.296826+0000", 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867546343000001, 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076706259999999998, 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 69 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296826+0000", 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296826+0000", 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296827+0000", 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.71e-07 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296949+0000", 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012259700000000001 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296964+0000", 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4651000000000001e-05 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296973+0000", 2024-09-17T17:29:51.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8550000000000008e-06 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.299602+0000", 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026291550000000002 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.299693+0000", 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1521999999999996e-05 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.299855+0000", 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000161817 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301550+0000", 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016943609999999999 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304409+0000", 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028592309999999998 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304449+0000", 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0623000000000001e-05 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304496+0000", 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6943000000000001e-05 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.296990+0000", 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867381838, 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075928840000000003, 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 70 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296990+0000", 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296990+0000", 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296991+0000", 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:29:51.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.296997+0000", 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5380000000000002e-06 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297007+0000", 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7629999999999995e-06 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297013+0000", 2024-09-17T17:29:51.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1360000000000002e-06 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.300309+0000", 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0032966720000000001 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.300363+0000", 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3461000000000002e-05 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.300454+0000", 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1452999999999993e-05 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301599+0000", 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011443720000000001 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304508+0000", 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002909029 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304539+0000", 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1778e-05 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304583+0000", 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3788999999999997e-05 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867347445, 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076388319999999999, 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 71 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.250 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297025+0000", 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5700000000000002e-07 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297030+0000", 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5430000000000004e-06 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297038+0000", 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4740000000000005e-06 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297045+0000", 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1630000000000001e-06 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301114+0000", 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040693709999999996 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301166+0000", 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1748e-05 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301255+0000", 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9326999999999994e-05 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304399+0000", 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031437930000000002 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304591+0000", 2024-09-17T17:29:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019223600000000001 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304625+0000", 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3560999999999999e-05 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304664+0000", 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8958999999999997e-05 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297056+0000", 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867315841, 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081331440000000001, 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 72 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.256 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297056+0000", 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297056+0000", 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297057+0000", 2024-09-17T17:29:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7399999999999998e-07 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297172+0000", 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011553900000000001 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297209+0000", 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6609000000000001e-05 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297219+0000", 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7480000000000005e-06 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301634+0000", 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044148679999999997 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301678+0000", 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4739e-05 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.301759+0000", 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1100000000000006e-05 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304714+0000", 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029548719999999999 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305111+0000", 2024-09-17T17:29:51.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000396876 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305145+0000", 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3424999999999999e-05 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305189+0000", 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4894000000000001e-05 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297253+0000", 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867118776000002, 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087998280000000009, 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 73 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297253+0000", 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297253+0000", 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297254+0000", 2024-09-17T17:29:51.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5600000000000005e-07 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297258+0000", 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2080000000000002e-06 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297266+0000", 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5190000000000003e-06 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297272+0000", 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7910000000000001e-06 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304008+0000", 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067363470000000002 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304061+0000", 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3219e-05 2024-09-17T17:29:51.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304142+0000", 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0717999999999995e-05 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305957+0000", 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018147510000000001 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305969+0000", 2024-09-17T17:29:51.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.252e-05 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306004+0000", 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4813e-05 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306053+0000", 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9286000000000002e-05 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867093236999999, 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093072850000000002, 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 74 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297279+0000", 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4200000000000001e-07 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297284+0000", 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8999999999999997e-06 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297289+0000", 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4580000000000003e-06 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297293+0000", 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7339999999999999e-06 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304762+0000", 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074683170000000004 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304811+0000", 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8992999999999997e-05 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.304894+0000", 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2805000000000004e-05 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306104+0000", 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012108119999999999 2024-09-17T17:29:51.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306502+0000", 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039784999999999999 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306541+0000", 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8779000000000003e-05 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306586+0000", 2024-09-17T17:29:51.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5194999999999997e-05 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.867071436, 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093839609999999997, 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.274 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 75 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297301+0000", 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297372+0000", 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1192000000000005e-05 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297381+0000", 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000003e-06 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297386+0000", 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7999999999999998e-06 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305232+0000", 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078461910000000006 2024-09-17T17:29:51.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305284+0000", 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2522999999999998e-05 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.305373+0000", 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8889999999999998e-05 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306598+0000", 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012251969999999999 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306630+0000", 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1718999999999999e-05 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306662+0000", 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1458999999999998e-05 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306685+0000", 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.304e-05 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.866972866000001, 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011397086000000001, 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 76 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297400+0000", 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7100000000000002e-07 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297403+0000", 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5920000000000001e-06 2024-09-17T17:29:51.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297410+0000", 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1489999999999997e-06 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297417+0000", 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-06 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306131+0000", 2024-09-17T17:29:51.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087138279999999999 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306185+0000", 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3783999999999999e-05 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306278+0000", 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3587999999999999e-05 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306994+0000", 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071571099999999995 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308729+0000", 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001735489 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308769+0000", 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9270000000000002e-05 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308796+0000", 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7603999999999999e-05 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:124 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.624911313999998, 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011177351, 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 124 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539467+0000", 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2089999999999996e-06 2024-09-17T17:29:51.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539473+0000", 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4690000000000002e-06 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539479+0000", 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545059+0000", 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055806400000000004 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545104+0000", 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5040999999999998e-05 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545172+0000", 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7249999999999995e-05 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545874+0000", 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070271799999999996 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550512+0000", 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046379530000000002 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550597+0000", 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025999999999995e-05 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550638+0000", 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0771e-05 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.as2 37/32 ECSubRead(tid=27, to_read={3:5ee91470:::smithi10238019-6:head=20480,4096,0}, subchunks={3:5ee91470:::smithi10238019-6:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.171345562, 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.671263006, 2024-09-17T17:29:51.292 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993028+0000", 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0219999999999999e-06 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993040+0000", 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1968e-05 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664125+0000", 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.67108543799999998 2024-09-17T17:29:51.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664134+0000", 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8640000000000002e-06 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664204+0000", 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9733999999999993e-05 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664290+0000", 2024-09-17T17:29:51.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.598e-05 2024-09-17T17:29:51.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:29:51.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:29:51.296 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:29:51.297 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:51.297 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:51.298 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:51.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:29:51.299 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:51.296+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:51.299 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:51.300 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:51.300 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:51.310 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:51.331 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:51.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:51.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:51.394 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:51.398 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:51.397+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:51.416 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:51.438 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:51.446 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:51.446 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:51.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:29:51.458 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:51.480 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:51.499 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:51.497+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:51.502 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:51.503 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:29:51.524 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:51.600 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:51.598+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:51.602 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:51.603 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:51.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:29:51.669 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:29:51.670 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:29:51.670 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-17T17:29:51.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-17T17:29:51.680 INFO:tasks.rados.rados.0.smithi102.stdout:82: done (2 left) 2024-09-17T17:29:51.680 INFO:tasks.rados.rados.0.smithi102.stdout:83: read oid 42 snap -1 2024-09-17T17:29:51.680 INFO:tasks.rados.rados.0.smithi102.stdout:83: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-17T17:29:51.680 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 40 current snap is 2 2024-09-17T17:29:51.680 INFO:tasks.rados.rados.0.smithi102.stdout:80: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-17T17:29:51.681 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:51.692 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 191490 2024-09-17T17:29:51.700 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:51.697+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:51.701 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 351913 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:78: finishing write tid 3 to smithi10238019-27 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:78: finishing write tid 4 to smithi10238019-27 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:78: oid 27 updating version 0 to 16 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:78: oid 27 version 16 is already newer than 14 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:78: oid 27 version 16 is already newer than 15 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 27 v 16 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:78: left oid 27 (ObjNum 57 snap 2 seq_num 57) 2024-09-17T17:29:51.706 INFO:tasks.rados.rados.0.smithi102.stdout:84: seq_num 60 ranges {671744=638976,1310720=376832} 2024-09-17T17:29:51.713 INFO:tasks.rados.rados.0.smithi102.stdout:84: writing smithi10238019-40 from 671744 to 1310720 tid 1 2024-09-17T17:29:51.715 INFO:tasks.rados.rados.0.smithi102.stdout:84: writing smithi10238019-40 from 1310720 to 1687552 tid 2 2024-09-17T17:29:51.715 INFO:tasks.rados.rados.0.smithi102.stdout:78: done (3 left) 2024-09-17T17:29:51.716 INFO:tasks.rados.rados.0.smithi102.stdout:80: done (2 left) 2024-09-17T17:29:51.716 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 31 current snap is 2 2024-09-17T17:29:51.716 INFO:tasks.rados.rados.0.smithi102.stdout:85: seq_num 61 ranges {32768=655360,688128=376832} 2024-09-17T17:29:51.719 INFO:tasks.rados.rados.0.smithi102.stdout:85: writing smithi10238019-31 from 32768 to 688128 tid 1 2024-09-17T17:29:51.722 INFO:tasks.rados.rados.0.smithi102.stdout:85: writing smithi10238019-31 from 688128 to 1064960 tid 2 2024-09-17T17:29:51.722 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 28 current snap is 2 2024-09-17T17:29:51.722 INFO:tasks.rados.rados.0.smithi102.stdout:86: seq_num 62 ranges {458752=671744,1130496=376832} 2024-09-17T17:29:51.726 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:51.726 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:51.728 INFO:tasks.rados.rados.0.smithi102.stdout:86: writing smithi10238019-28 from 458752 to 1130496 tid 1 2024-09-17T17:29:51.730 INFO:tasks.rados.rados.0.smithi102.stdout:86: writing smithi10238019-28 from 1130496 to 1507328 tid 2 2024-09-17T17:29:51.730 INFO:tasks.rados.rados.0.smithi102.stdout:87: read oid 33 snap -1 2024-09-17T17:29:51.730 INFO:tasks.rados.rados.0.smithi102.stdout:87: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-17T17:29:51.730 INFO:tasks.rados.rados.0.smithi102.stdout:88: setattr oid 41 current snap is 2 2024-09-17T17:29:51.731 INFO:tasks.rados.rados.0.smithi102.stdout:83: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-17T17:29:51.731 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 22917 2024-09-17T17:29:51.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:29:51.740 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 397985 2024-09-17T17:29:51.747 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 695036 2024-09-17T17:29:51.747 INFO:tasks.rados.rados.0.smithi102.stdout:84: finishing write tid 1 to smithi10238019-40 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:84: finishing write tid 2 to smithi10238019-40 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:85: finishing write tid 1 to smithi10238019-31 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:85: finishing write tid 2 to smithi10238019-31 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:86: finishing write tid 1 to smithi10238019-28 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:85: finishing write tid 3 to smithi10238019-31 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:86: finishing write tid 2 to smithi10238019-28 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:85: finishing write tid 4 to smithi10238019-31 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:85: oid 31 updating version 0 to 15 2024-09-17T17:29:51.748 INFO:tasks.rados.rados.0.smithi102.stdout:85: oid 31 updating version 15 to 16 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:85: oid 31 updating version 16 to 17 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 17 (ObjNum 61 snap 2 seq_num 61) dirty exists 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:85: left oid 31 (ObjNum 61 snap 2 seq_num 61) 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: finishing write tid 3 to smithi10238019-28 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: finishing write tid 4 to smithi10238019-28 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: oid 28 updating version 0 to 24 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: oid 28 updating version 24 to 26 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: oid 28 version 26 is already newer than 25 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 28 v 26 (ObjNum 62 snap 2 seq_num 62) dirty exists 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: left oid 28 (ObjNum 62 snap 2 seq_num 62) 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:83: done (5 left) 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:85: done (4 left) 2024-09-17T17:29:51.749 INFO:tasks.rados.rados.0.smithi102.stdout:86: done (3 left) 2024-09-17T17:29:51.750 INFO:tasks.rados.rados.0.smithi102.stdout:89: snap_create 2024-09-17T17:29:51.801 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:51.799+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:51.844 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:51.844 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:51.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:29:51.902 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:51.900+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:29:51.959 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:51.959 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:51.963 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 41 v 11 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-17T17:29:51.968 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:29:52.004 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:52.002+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:52.075 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.075 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.083 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:29:52.105 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:52.103+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:52.199 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.199 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.205 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:52.204+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:52.208 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:29:52.245 INFO:tasks.rados.rados.0.smithi102.stdout:87: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-17T17:29:52.248 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 499986 2024-09-17T17:29:52.249 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:52.258 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 32408 2024-09-17T17:29:52.306 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:52.304+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:29:52.318 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.318 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.326 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:29:52.407 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:52.405+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:52.470 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.470 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.478 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:29:52.487 INFO:tasks.rados.rados.0.smithi102.stdout:84: finishing write tid 3 to smithi10238019-40 2024-09-17T17:29:52.508 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:52.506+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:52.581 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.582 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.590 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:29:52.594 INFO:tasks.rados.rados.0.smithi102.stdout:84: finishing write tid 4 to smithi10238019-40 2024-09-17T17:29:52.594 INFO:tasks.rados.rados.0.smithi102.stdout:84: oid 40 updating version 0 to 5 2024-09-17T17:29:52.594 INFO:tasks.rados.rados.0.smithi102.stdout:84: oid 40 updating version 5 to 7 2024-09-17T17:29:52.594 INFO:tasks.rados.rados.0.smithi102.stdout:84: oid 40 version 7 is already newer than 6 2024-09-17T17:29:52.594 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 40 v 7 (ObjNum 60 snap 2 seq_num 60) dirty exists 2024-09-17T17:29:52.594 INFO:tasks.rados.rados.0.smithi102.stdout:84: left oid 40 (ObjNum 60 snap 2 seq_num 60) 2024-09-17T17:29:52.609 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:52.606+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:52.678 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T17:29:52.676+0000 7f873d521640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-17T17:29:52.682 INFO:teuthology.orchestra.run.smithi007.stderr:osd set-full-ratio 0.001 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:84: done (3 left) 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:87: done (2 left) 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:88: done (1 left) 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:89: done (0 left) 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:90: read oid 35 snap 2 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:90: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 43 current snap is 3 2024-09-17T17:29:52.686 INFO:tasks.rados.rados.0.smithi102.stdout:91: seq_num 63 ranges {720896=688128,1409024=376832} 2024-09-17T17:29:52.694 INFO:tasks.rados.rados.0.smithi102.stdout:91: writing smithi10238019-43 from 720896 to 1409024 tid 1 2024-09-17T17:29:52.696 INFO:tasks.rados.rados.0.smithi102.stdout:91: writing smithi10238019-43 from 1409024 to 1785856 tid 2 2024-09-17T17:29:52.696 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 48 current snap is 3 2024-09-17T17:29:52.696 INFO:tasks.rados.rados.0.smithi102.stdout:90: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-17T17:29:52.696 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:52.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:29:52.702 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.702 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.702 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:52.707 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:52.709 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:52.707+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:52.711 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:29:52.713 INFO:tasks.rados.rados.0.smithi102.stdout:92: seq_num 64 ranges {802816=704512,1507328=376832} 2024-09-17T17:29:52.721 INFO:tasks.rados.rados.0.smithi102.stdout:92: writing smithi10238019-48 from 802816 to 1507328 tid 1 2024-09-17T17:29:52.724 INFO:tasks.rados.rados.0.smithi102.stdout:92: writing smithi10238019-48 from 1507328 to 1884160 tid 2 2024-09-17T17:29:52.724 INFO:tasks.rados.rados.0.smithi102.stdout:90: done (2 left) 2024-09-17T17:29:52.724 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 24 current snap is 3 2024-09-17T17:29:52.724 INFO:tasks.rados.rados.0.smithi102.stdout:91: finishing write tid 1 to smithi10238019-43 2024-09-17T17:29:52.724 INFO:tasks.rados.rados.0.smithi102.stdout:91: finishing write tid 2 to smithi10238019-43 2024-09-17T17:29:52.724 INFO:tasks.rados.rados.0.smithi102.stdout:91: finishing write tid 3 to smithi10238019-43 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:91: finishing write tid 4 to smithi10238019-43 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:91: oid 43 updating version 0 to 19 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:91: oid 43 version 19 is already newer than 17 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:91: oid 43 version 19 is already newer than 18 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 43 v 19 (ObjNum 63 snap 3 seq_num 63) dirty exists 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:91: left oid 43 (ObjNum 63 snap 3 seq_num 63) 2024-09-17T17:29:52.725 INFO:tasks.rados.rados.0.smithi102.stdout:93: seq_num 65 ranges {393216=720896,1114112=376832} 2024-09-17T17:29:52.729 INFO:tasks.rados.rados.0.smithi102.stdout:93: writing smithi10238019-24 from 393216 to 1114112 tid 1 2024-09-17T17:29:52.732 INFO:tasks.rados.rados.0.smithi102.stdout:93: writing smithi10238019-24 from 1114112 to 1490944 tid 2 2024-09-17T17:29:52.732 INFO:tasks.rados.rados.0.smithi102.stdout:91: done (2 left) 2024-09-17T17:29:52.732 INFO:tasks.rados.rados.0.smithi102.stdout:94: delete oid 45 current snap is 3 2024-09-17T17:29:52.732 INFO:tasks.rados.rados.0.smithi102.stdout:92: finishing write tid 1 to smithi10238019-48 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: finishing write tid 2 to smithi10238019-48 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: finishing write tid 3 to smithi10238019-48 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: finishing write tid 4 to smithi10238019-48 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: oid 48 updating version 0 to 19 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: oid 48 updating version 19 to 20 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: oid 48 updating version 20 to 21 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 48 v 21 (ObjNum 64 snap 3 seq_num 64) dirty exists 2024-09-17T17:29:52.733 INFO:tasks.rados.rados.0.smithi102.stdout:92: left oid 48 (ObjNum 64 snap 3 seq_num 64) 2024-09-17T17:29:52.739 INFO:tasks.rados.rados.0.smithi102.stdout:93: finishing write tid 1 to smithi10238019-24 2024-09-17T17:29:52.739 INFO:tasks.rados.rados.0.smithi102.stdout:92: done (2 left) 2024-09-17T17:29:52.739 INFO:tasks.rados.rados.0.smithi102.stdout:94: done (1 left) 2024-09-17T17:29:52.739 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 37 current snap is 3 2024-09-17T17:29:52.739 INFO:tasks.rados.rados.0.smithi102.stdout:95: seq_num 67 ranges {606208=753664,1359872=376832} 2024-09-17T17:29:52.744 INFO:tasks.rados.rados.0.smithi102.stdout:95: writing smithi10238019-37 from 606208 to 1359872 tid 1 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:95: writing smithi10238019-37 from 1359872 to 1736704 tid 2 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:96: snap_create 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:93: finishing write tid 2 to smithi10238019-24 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:93: finishing write tid 3 to smithi10238019-24 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:93: finishing write tid 4 to smithi10238019-24 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:93: oid 24 updating version 0 to 5 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:93: oid 24 version 5 is already newer than 4 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:93: oid 24 updating version 5 to 6 2024-09-17T17:29:52.747 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 24 v 6 (ObjNum 65 snap 3 seq_num 65) dirty exists 2024-09-17T17:29:52.748 INFO:tasks.rados.rados.0.smithi102.stdout:93: left oid 24 (ObjNum 65 snap 3 seq_num 65) 2024-09-17T17:29:52.749 INFO:tasks.rados.rados.0.smithi102.stdout:95: finishing write tid 1 to smithi10238019-37 2024-09-17T17:29:52.750 INFO:tasks.rados.rados.0.smithi102.stdout:95: finishing write tid 2 to smithi10238019-37 2024-09-17T17:29:52.750 INFO:tasks.rados.rados.0.smithi102.stdout:95: finishing write tid 3 to smithi10238019-37 2024-09-17T17:29:52.751 INFO:tasks.rados.rados.0.smithi102.stdout:95: finishing write tid 4 to smithi10238019-37 2024-09-17T17:29:52.751 INFO:tasks.rados.rados.0.smithi102.stdout:95: oid 37 updating version 0 to 8 2024-09-17T17:29:52.751 INFO:tasks.rados.rados.0.smithi102.stdout:95: oid 37 updating version 8 to 10 2024-09-17T17:29:52.752 INFO:tasks.rados.rados.0.smithi102.stdout:95: oid 37 version 10 is already newer than 9 2024-09-17T17:29:52.752 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 37 v 10 (ObjNum 67 snap 3 seq_num 67) dirty exists 2024-09-17T17:29:52.752 INFO:tasks.rados.rados.0.smithi102.stdout:95: left oid 37 (ObjNum 67 snap 3 seq_num 67) 2024-09-17T17:29:52.813 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:52.811+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:52.820 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.820 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.829 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:29:52.914 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:29:52.912+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:29:52.972 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:52.972 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:52.980 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:29:53.015 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:53.013+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:53.091 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:29:53.135 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:53.132+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:53.195 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:53.195 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:53.203 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:29:53.217 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:29:53.215+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:29:53.318 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:53.316+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:53.347 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:53.347 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:53.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:29:53.418 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:53.416+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:53.460 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:53.460 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:53.468 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:29:53.519 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:53.517+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:53.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:29:53.612 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:53.612 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:53.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:29:53.621 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:53.618+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:53.682 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:53.682 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:29:53.689 INFO:tasks.rados.rados.0.smithi102.stdout:93: done (2 left) 2024-09-17T17:29:53.690 INFO:tasks.rados.rados.0.smithi102.stdout:95: done (1 left) 2024-09-17T17:29:53.690 INFO:tasks.rados.rados.0.smithi102.stdout:96: done (0 left) 2024-09-17T17:29:53.690 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 16 current snap is 4 2024-09-17T17:29:53.690 INFO:tasks.rados.rados.0.smithi102.stdout:97: seq_num 68 ranges {262144=770048,1032192=376832} 2024-09-17T17:29:53.696 INFO:tasks.rados.rados.0.smithi102.stdout:97: writing smithi10238019-16 from 262144 to 1032192 tid 1 2024-09-17T17:29:53.698 INFO:tasks.rados.rados.0.smithi102.stdout:97: writing smithi10238019-16 from 1032192 to 1409024 tid 2 2024-09-17T17:29:53.698 INFO:tasks.rados.rados.0.smithi102.stdout:98: read oid 44 snap -1 2024-09-17T17:29:53.698 INFO:tasks.rados.rados.0.smithi102.stdout:98: expect (ObjNum 55 snap 2 seq_num 55) 2024-09-17T17:29:53.698 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 41 current snap is 4 2024-09-17T17:29:53.698 INFO:tasks.rados.rados.0.smithi102.stdout:99: seq_num 69 ranges {688128=786432,1474560=376832} 2024-09-17T17:29:53.702 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-17T17:29:53.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-17T17:29:53.706 INFO:tasks.rados.rados.0.smithi102.stdout:99: writing smithi10238019-41 from 688128 to 1474560 tid 1 2024-09-17T17:29:53.708 INFO:tasks.rados.rados.0.smithi102.stdout:99: writing smithi10238019-41 from 1474560 to 1851392 tid 2 2024-09-17T17:29:53.709 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 23 current snap is 4 2024-09-17T17:29:53.709 INFO:tasks.rados.rados.0.smithi102.stdout:97: finishing write tid 1 to smithi10238019-16 2024-09-17T17:29:53.709 INFO:tasks.rados.rados.0.smithi102.stdout:100: seq_num 70 ranges {1294336=802816,2097152=376832} 2024-09-17T17:29:53.717 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:53.717 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:53.719 INFO:tasks.rados.rados.0.smithi102.stdout:100: writing smithi10238019-23 from 1294336 to 2097152 tid 1 2024-09-17T17:29:53.720 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:53.718+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:100: writing smithi10238019-23 from 2097152 to 2473984 tid 2 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:101: copy_from oid 28 from oid 24 current snap is 4 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: finishing write tid 2 to smithi10238019-16 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: finishing write tid 3 to smithi10238019-16 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: finishing write tid 4 to smithi10238019-16 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: oid 16 updating version 0 to 7 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: oid 16 version 7 is already newer than 6 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: oid 16 updating version 7 to 8 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 8 (ObjNum 68 snap 4 seq_num 68) dirty exists 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:97: left oid 16 (ObjNum 68 snap 4 seq_num 68) 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:98: expect (ObjNum 55 snap 2 seq_num 55) 2024-09-17T17:29:53.723 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:53.731 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:53.740 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:53.741 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:53.741 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:53.744 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:53.744 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:53.744 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:29:53.744 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:53.749 INFO:tasks.rados.rados.0.smithi102.stdout:99: finishing write tid 1 to smithi10238019-41 2024-09-17T17:29:53.749 INFO:tasks.rados.rados.0.smithi102.stdout:99: finishing write tid 2 to smithi10238019-41 2024-09-17T17:29:53.749 INFO:tasks.rados.rados.0.smithi102.stdout:99: finishing write tid 3 to smithi10238019-41 2024-09-17T17:29:53.749 INFO:tasks.rados.rados.0.smithi102.stdout:99: finishing write tid 4 to smithi10238019-41 2024-09-17T17:29:53.749 INFO:tasks.rados.rados.0.smithi102.stdout:99: oid 41 updating version 0 to 13 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:99: oid 41 updating version 13 to 14 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:99: oid 41 updating version 14 to 15 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 41 v 15 (ObjNum 69 snap 4 seq_num 69) dirty exists 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:99: left oid 41 (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:100: finishing write tid 1 to smithi10238019-23 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:100: finishing write tid 2 to smithi10238019-23 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:101: finishing copy_from racing read to smithi10238019-28 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:101: finishing copy_from to smithi10238019-28 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 28 v 6 (ObjNum 65 snap 3 seq_num 65) dirty exists 2024-09-17T17:29:53.750 INFO:tasks.rados.rados.0.smithi102.stdout:97: done (4 left) 2024-09-17T17:29:53.751 INFO:tasks.rados.rados.0.smithi102.stdout:98: done (3 left) 2024-09-17T17:29:53.751 INFO:tasks.rados.rados.0.smithi102.stdout:99: done (2 left) 2024-09-17T17:29:53.751 INFO:tasks.rados.rados.0.smithi102.stdout:101: done (1 left) 2024-09-17T17:29:53.751 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 46 current snap is 4 2024-09-17T17:29:53.751 INFO:tasks.rados.rados.0.smithi102.stdout:102: seq_num 71 ranges {770048=425984,1196032=573440,1769472=196608} 2024-09-17T17:29:53.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:29:53.755 INFO:tasks.rados.rados.0.smithi102.stdout:102: writing smithi10238019-46 from 770048 to 1196032 tid 1 2024-09-17T17:29:53.758 INFO:tasks.rados.rados.0.smithi102.stdout:102: writing smithi10238019-46 from 1196032 to 1769472 tid 2 2024-09-17T17:29:53.759 INFO:tasks.rados.rados.0.smithi102.stdout:102: writing smithi10238019-46 from 1769472 to 1966080 tid 3 2024-09-17T17:29:53.759 INFO:tasks.rados.rados.0.smithi102.stdout:103: setattr oid 17 current snap is 4 2024-09-17T17:29:53.760 INFO:tasks.rados.rados.0.smithi102.stdout:102: finishing write tid 1 to smithi10238019-46 2024-09-17T17:29:53.761 INFO:tasks.rados.rados.0.smithi102.stdout:104: read oid 10 snap -1 2024-09-17T17:29:53.762 INFO:tasks.rados.rados.0.smithi102.stdout:104: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-17T17:29:53.762 INFO:tasks.rados.rados.0.smithi102.stdout:105: read oid 27 snap 3 2024-09-17T17:29:53.762 INFO:tasks.rados.rados.0.smithi102.stdout:105: expect (ObjNum 57 snap 2 seq_num 57) 2024-09-17T17:29:53.762 INFO:tasks.rados.rados.0.smithi102.stdout:106: delete oid 47 current snap is 4 2024-09-17T17:29:53.762 INFO:tasks.rados.rados.0.smithi102.stdout:102: finishing write tid 2 to smithi10238019-46 2024-09-17T17:29:53.764 INFO:tasks.rados.rados.0.smithi102.stdout:102: finishing write tid 3 to smithi10238019-46 2024-09-17T17:29:53.766 INFO:tasks.rados.rados.0.smithi102.stdout:102: finishing write tid 4 to smithi10238019-46 2024-09-17T17:29:53.766 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:53.767 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:53.767 INFO:tasks.rados.rados.0.smithi102.stdout:104: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-17T17:29:53.767 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 89826 2024-09-17T17:29:53.768 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:53.770 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 144889 2024-09-17T17:29:53.771 INFO:tasks.rados.rados.0.smithi102.stdout:102: finishing write tid 5 to smithi10238019-46 2024-09-17T17:29:53.771 INFO:tasks.rados.rados.0.smithi102.stdout:102: oid 46 updating version 0 to 15 2024-09-17T17:29:53.771 INFO:tasks.rados.rados.0.smithi102.stdout:102: oid 46 version 15 is already newer than 12 2024-09-17T17:29:53.771 INFO:tasks.rados.rados.0.smithi102.stdout:102: oid 46 version 15 is already newer than 14 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:102: oid 46 version 15 is already newer than 13 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 15 (ObjNum 71 snap 4 seq_num 71) dirty exists 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:102: left oid 46 (ObjNum 71 snap 4 seq_num 71) 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 17 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:102: done (5 left) 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:103: done (4 left) 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:104: done (3 left) 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:106: done (2 left) 2024-09-17T17:29:53.772 INFO:tasks.rados.rados.0.smithi102.stdout:107: snap_remove snap 2 2024-09-17T17:29:53.788 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:53.788 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:53.809 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:53.821 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:53.820+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:53.888 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:53.888 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:53.888 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:29:53.888 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:29:53.889 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:53.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:29:53.922 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:53.920+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:53.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:53.966 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:53.987 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:54.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:54.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:54.023 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:54.021+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:54.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:50 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'32, mlcod=24'21)", 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:29.992613+0000", 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.052663259999999, 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011526900000000001, 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992613+0000", 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992613+0000", 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992613+0000", 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992686+0000", 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2521000000000006e-05 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992703+0000", 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7714e-05 2024-09-17T17:29:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992707+0000", 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4910000000000001e-06 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992756+0000", 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9424e-05 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.992779+0000", 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2917000000000001e-05 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.993731+0000", 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095166000000000001 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.993765+0000", 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4564999999999999e-05 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:54 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'36, mlcod=24'33)", 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:29.999837+0000", 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.045439335000001, 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00111947, 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.999837+0000", 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.999837+0000", 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.999837+0000", 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-07 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.999915+0000", 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8232999999999994e-05 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.999937+0000", 2024-09-17T17:29:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2016000000000001e-05 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:29.999941+0000", 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.000068+0000", 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012645399999999999 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.000090+0000", 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2470999999999999e-05 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.000893+0000", 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080330199999999999 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.000956+0000", 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2708999999999999e-05 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:89 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'63, mlcod=24'60)", 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:30.401018+0000", 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.644257363000001, 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001067946, 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401018+0000", 2024-09-17T17:29:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401018+0000", 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401020+0000", 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401028+0000", 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.636e-06 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401073+0000", 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4227000000000002e-05 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401081+0000", 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8789999999999992e-06 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401353+0000", 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027125000000000001 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401366+0000", 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3603e-05 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.402067+0000", 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00070111299999999995 2024-09-17T17:29:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.402086+0000", 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9080999999999999e-05 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:90 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'64, mlcod=24'60)", 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:30.401327+0000", 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.643949036999999, 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010895200000000001, 2024-09-17T17:29:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401327+0000", 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401327+0000", 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401327+0000", 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401383+0000", 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5123000000000001e-05 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401411+0000", 2024-09-17T17:29:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8844999999999999e-05 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401417+0000", 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9710000000000001e-06 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401537+0000", 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119577 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.401549+0000", 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1602e-05 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.402394+0000", 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00084552100000000001 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.402416+0000", 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2297999999999999e-05 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:112 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'82, mlcod=24'80)", 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:30.538950+0000", 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.506325469, 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001342563, 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.538950+0000", 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.538950+0000", 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.538952+0000", 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9429999999999999e-06 2024-09-17T17:29:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.538966+0000", 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3419e-05 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539051+0000", 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4952999999999998e-05 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539069+0000", 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8612999999999999e-05 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539144+0000", 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4449999999999994e-05 2024-09-17T17:29:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539167+0000", 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3529000000000001e-05 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.540257+0000", 2024-09-17T17:29:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010897630000000001 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.540293+0000", 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5892999999999997e-05 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:113 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'83, mlcod=24'80)", 2024-09-17T17:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:30.539202+0000", 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.506074247000001, 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011543269999999999, 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539202+0000", 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539202+0000", 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539202+0000", 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539207+0000", 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6959999999999996e-06 2024-09-17T17:29:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539238+0000", 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1133999999999997e-05 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539245+0000", 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4910000000000002e-06 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539511+0000", 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026622700000000001 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.539546+0000", 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5017e-05 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.540328+0000", 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078176199999999997 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:30.540356+0000", 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8379000000000001e-05 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3616641459999999, 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020160569999999999, 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0499999999999999e-07 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683889+0000", 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000276748 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683926+0000", 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7307e-05 2024-09-17T17:29:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683937+0000", 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0445000000000001e-05 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684015+0000", 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8745999999999994e-05 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684028+0000", 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2733e-05 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015570569999999999 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2616e-05 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpRead(3.0s3 39/35 ECSubRead(tid=26, to_read={3:07ceeeba:::smithi10238019-36:head=0,147456,0}, subchunks={3:07ceeeba:::smithi10238019-36:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.359690735, 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001493276, 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2700000000000001e-07 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685594+0000", 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6400000000000003e-06 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685681+0000", 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472999999999994e-05 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685691+0000", 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020000000000001e-05 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685761+0000", 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9029000000000006e-05 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687078+0000", 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001317887 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 39/35 ECSubWrite(tid=48, reqid=client.4566.0:188, at_version=39'20, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685900+0000", 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.359375607, 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001607336, 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685900+0000", 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685900+0000", 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685901+0000", 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686027+0000", 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126308 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686073+0000", 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5893999999999998e-05 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686083+0000", 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7610000000000002e-06 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686153+0000", 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9740999999999995e-05 2024-09-17T17:29:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686164+0000", 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1146e-05 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001303284 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687508+0000", 2024-09-17T17:29:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0250999999999998e-05 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.353391824, 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018516470000000001, 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691889+0000", 2024-09-17T17:29:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6330000000000004e-06 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692145+0000", 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025631599999999998 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692190+0000", 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4969000000000003e-05 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692199+0000", 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4599999999999992e-06 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692277+0000", 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7676999999999995e-05 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692289+0000", 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693695+0000", 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014064679999999999 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693736+0000", 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-05 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 39/35 ECSubWrite(tid=51, reqid=client.4566.0:196, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.693817+0000", 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3514585769999998, 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018183520000000001, 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693817+0000", 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693817+0000", 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693818+0000", 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3799999999999996e-07 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694010+0000", 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000192058 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694049+0000", 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.888e-05 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694057+0000", 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3520000000000007e-06 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694125+0000", 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7973999999999997e-05 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694139+0000", 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3295999999999999e-05 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695600+0000", 2024-09-17T17:29:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001461279 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695636+0000", 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5775000000000002e-05 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 39/35 ECSubWrite(tid=54, reqid=client.4566.0:197, at_version=39'8, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.694706+0000", 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3505697699999999, 2024-09-17T17:29:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301361, 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694706+0000", 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694706+0000", 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694707+0000", 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2900000000000002e-07 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694724+0000", 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7373000000000001e-05 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694762+0000", 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7227000000000001e-05 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694770+0000", 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694830+0000", 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9911999999999998e-05 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694840+0000", 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0288999999999999e-05 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695966+0000", 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011258430000000001 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.696007+0000", 2024-09-17T17:29:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1177999999999998e-05 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 39/32 ECSubWrite(tid=15, reqid=client.4566.0:199, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.345059987, 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001846723, 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700460+0000", 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024369999999999999 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700559+0000", 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8998000000000003e-05 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700569+0000", 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9510000000000001e-06 2024-09-17T17:29:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700641+0000", 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2325999999999998e-05 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700656+0000", 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702016+0000", 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013601799999999999 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702063+0000", 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6384999999999997e-05 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 39/35 ECSubWrite(tid=28, reqid=client.4566.0:212, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.711106+0000", 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3341701079999999, 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012324689999999999, 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711106+0000", 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711106+0000", 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711107+0000", 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6700000000000002e-07 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711138+0000", 2024-09-17T17:29:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1393999999999998e-05 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711196+0000", 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8118999999999997e-05 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711205+0000", 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9590000000000001e-06 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711284+0000", 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9258999999999994e-05 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711297+0000", 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2082000000000001e-05 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712294+0000", 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099769000000000008 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712338+0000", 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3998999999999997e-05 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=80, reqid=client.4566.0:274, at_version=42'13, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.33735752400000002, 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003864363, 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707919+0000", 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708479+0000", 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055997299999999996 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708513+0000", 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4359000000000001e-05 2024-09-17T17:29:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708524+0000", 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0346999999999999e-05 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708600+0000", 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999997e-05 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708619+0000", 2024-09-17T17:29:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8490000000000001e-05 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711752+0000", 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031327600000000001 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711783+0000", 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1035999999999999e-05 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=81, reqid=client.4566.0:275, at_version=42'14, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.33498971900000002, 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025720560000000001, 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710287+0000", 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3300000000000001e-07 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710492+0000", 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020486500000000001 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710535+0000", 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3510999999999997e-05 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710545+0000", 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9869999999999994e-06 2024-09-17T17:29:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710609+0000", 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3379000000000004e-05 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710619+0000", 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0098000000000001e-05 2024-09-17T17:29:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712823+0000", 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022037350000000001 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712858+0000", 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5648000000000002e-05 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=82, reqid=client.4566.0:276, at_version=42'15, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.711421+0000", 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.33385463199999998, 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001452769, 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711421+0000", 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711421+0000", 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711422+0000", 2024-09-17T17:29:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711443+0000", 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0661000000000001e-05 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711478+0000", 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4921e-05 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711497+0000", 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9786999999999999e-05 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711555+0000", 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7343999999999998e-05 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711565+0000", 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0169999999999999e-05 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712862+0000", 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001296836 2024-09-17T17:29:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712874+0000", 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2204999999999999e-05 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 42/32 ECSubWrite(tid=32, reqid=client.4566.0:278, at_version=42'23, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.32215418299999998, 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001824928, 2024-09-17T17:29:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2800000000000002e-07 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723213+0000", 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0434999999999997e-05 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723240+0000", 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7090999999999999e-05 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723250+0000", 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0141e-05 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723328+0000", 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8588e-05 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723341+0000", 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2568000000000001e-05 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.724879+0000", 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015376599999999999 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.724947+0000", 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8016999999999996e-05 2024-09-17T17:29:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 42/35 ECSubWrite(tid=86, reqid=client.4566.0:282, at_version=42'30, trim_to=0'0, roll_forward_to=41'28))", 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.31157895000000002, 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002415295, 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733698+0000", 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2759999999999999e-06 2024-09-17T17:29:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734149+0000", 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045118399999999998 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734186+0000", 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7109e-05 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734195+0000", 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0539999999999995e-06 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734270+0000", 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5165000000000003e-05 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734282+0000", 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2763e-05 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736069+0000", 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017863530000000001 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736112+0000", 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3390999999999998e-05 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 42/32 ECSubWrite(tid=16, reqid=client.4566.0:289, at_version=42'12, trim_to=0'0, roll_forward_to=41'10))", 2024-09-17T17:29:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.762220+0000", 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.28305552099999998, 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001441988, 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762220+0000", 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762220+0000", 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762221+0000", 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762424+0000", 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000203069 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762454+0000", 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.972e-05 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762463+0000", 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6290000000000005e-06 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762536+0000", 2024-09-17T17:29:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2893999999999997e-05 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762547+0000", 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0628999999999999e-05 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.763626+0000", 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010797999999999999 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.763662+0000", 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5833000000000001e-05 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.143 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:54.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:54.144 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:54.141+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:54.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:29:54.146 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:54.147 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:29:54.114+0000 7fbd5ffff640 0 -- 172.21.15.7:0/715906822 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fbd6821f290 msgr2=0x7fbd68221680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:29:54.148 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:54.148 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:54.245 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:54.243+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:54.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:29:54.302 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:54.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:29:54.346 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:54.344+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:54.416 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:54.417 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:54.417 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:29:54.417 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:29:54.417 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:54.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:29:54.448 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:54.446+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:54.533 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:54.533 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:29:54.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:29:54.533 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:29:54.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/35 ECSubWrite(tid=8, reqid=client.4566.0:179, at_version=37'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:48.990934+0000", 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5396597700000001, 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010000580000000001, 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.990934+0000", 2024-09-17T17:29:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.990934+0000", 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.990935+0000", 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.98e-07 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.990950+0000", 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4993e-05 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.990979+0000", 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.953e-05 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.990988+0000", 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0089000000000001e-05 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991046+0000", 2024-09-17T17:29:54.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4649999999999997e-06 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991893+0000", 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00084701999999999998 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991935+0000", 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1872999999999997e-05 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/32 ECSubWrite(tid=9, reqid=client.4566.0:176, at_version=37'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:48.991388+0000", 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5392059170000003, 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00097991099999999998, 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991388+0000", 2024-09-17T17:29:54.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991388+0000", 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991389+0000", 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991406+0000", 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7068e-05 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991439+0000", 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3086999999999997e-05 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991447+0000", 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8830000000000006e-06 2024-09-17T17:29:54.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991499+0000", 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2000999999999999e-05 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991508+0000", 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1400000000000006e-06 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.992335+0000", 2024-09-17T17:29:54.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082759100000000003 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.992368+0000", 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2858999999999998e-05 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=14, reqid=client.4566.0:184, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8520879969999999, 2024-09-17T17:29:54.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001332981, 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6999999999999999e-07 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678530+0000", 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3696000000000001e-05 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678576+0000", 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5528999999999997e-05 2024-09-17T17:29:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678588+0000", 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2527999999999999e-05 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678664+0000", 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5785999999999996e-05 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678678+0000", 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.436e-05 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.679792+0000", 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001113246 2024-09-17T17:29:54.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.679839+0000", 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7666000000000001e-05 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:29:54.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8457187089999998, 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002820571, 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684890+0000", 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3499000000000001e-05 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685111+0000", 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022111 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685121+0000", 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7070000000000004e-06 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685199+0000", 2024-09-17T17:29:54.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8145999999999993e-05 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685209+0000", 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0138e-05 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687657+0000", 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024481260000000001 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687696+0000", 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.909e-05 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/35 ECSubWrite(tid=11, reqid=osd.1.0:10, at_version=0'0, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8451460499999999, 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002246264, 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.554 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:29:54.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:29:54.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685449+0000", 2024-09-17T17:29:54.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0800000000000004e-07 2024-09-17T17:29:54.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685461+0000", 2024-09-17T17:29:54.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1664999999999999e-05 2024-09-17T17:29:54.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685533+0000", 2024-09-17T17:29:54.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2149000000000003e-05 2024-09-17T17:29:54.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685542+0000", 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685619+0000", 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7186000000000003e-05 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687653+0000", 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020252030000000002 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687694+0000", 2024-09-17T17:29:54.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1072000000000002e-05 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 39/32 ECSubWrite(tid=13, reqid=osd.14.0:13, at_version=39'10, trim_to=0'0, roll_forward_to=37'8))", 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691662+0000", 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8389317570000001, 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011926249999999999, 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.564 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691662+0000", 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691662+0000", 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691663+0000", 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.84e-07 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691694+0000", 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609000000000002e-05 2024-09-17T17:29:54.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691741+0000", 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6326999999999999e-05 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691751+0000", 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691831+0000", 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0129999999999993e-05 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691844+0000", 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2903e-05 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692817+0000", 2024-09-17T17:29:54.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00097266099999999999 2024-09-17T17:29:54.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692855+0000", 2024-09-17T17:29:54.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8040999999999997e-05 2024-09-17T17:29:54.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.838631248, 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022546670000000001, 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8700000000000002e-07 2024-09-17T17:29:54.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692554+0000", 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000590721 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692590+0000", 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6350000000000003e-05 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692602+0000", 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692708+0000", 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106835 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692718+0000", 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694181+0000", 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014632149999999999 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694218+0000", 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6730999999999998e-05 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=51, reqid=client.4566.0:196, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.693896+0000", 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8366986230000002, 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016845129999999999, 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693896+0000", 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693896+0000", 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693897+0000", 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0079999999999999e-06 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694141+0000", 2024-09-17T17:29:54.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000244856 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694178+0000", 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6096999999999997e-05 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694186+0000", 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8740000000000001e-06 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694255+0000", 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8415000000000001e-05 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694264+0000", 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9400000000000008e-06 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695539+0000", 2024-09-17T17:29:54.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001274982 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695580+0000", 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1341000000000003e-05 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=54, reqid=client.4566.0:197, at_version=39'8, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.694726+0000", 2024-09-17T17:29:54.580 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8358679059999998, 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010910270000000001, 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694726+0000", 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694726+0000", 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694727+0000", 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2999999999999999e-07 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694746+0000", 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9228e-05 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694783+0000", 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6044000000000003e-05 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694792+0000", 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3370000000000006e-06 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694858+0000", 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5899e-05 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694868+0000", 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0372e-05 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695783+0000", 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091487700000000003 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695817+0000", 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.434e-05 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 39/32 ECSubWrite(tid=14, reqid=osd.14.0:13, at_version=0'0, trim_to=0'0, roll_forward_to=39'10))", 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698072+0000", 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8325223419999999, 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013017510000000001, 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698072+0000", 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698072+0000", 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698073+0000", 2024-09-17T17:29:54.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7700000000000004e-07 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698082+0000", 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1160000000000005e-06 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698109+0000", 2024-09-17T17:29:54.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6973000000000001e-05 2024-09-17T17:29:54.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698117+0000", 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3259999999999996e-06 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698169+0000", 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2083000000000003e-05 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698177+0000", 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4340000000000003e-06 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699332+0000", 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011553259999999999 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699374+0000", 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1715999999999998e-05 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=55, reqid=client.4566.0:197, at_version=0'0, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698827+0000", 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.831766816, 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001081959, 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698827+0000", 2024-09-17T17:29:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698827+0000", 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698828+0000", 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8100000000000001e-07 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698841+0000", 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2680999999999999e-05 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698877+0000", 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5553000000000003e-05 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698885+0000", 2024-09-17T17:29:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7700000000000007e-06 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698973+0000", 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8077000000000004e-05 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698983+0000", 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.679e-06 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699862+0000", 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087841000000000004 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699909+0000", 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7808e-05 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/35 ECSubWrite(tid=12, reqid=client.4566.0:203, at_version=39'9, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:29:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.705385+0000", 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8252090719999998, 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017198960000000001, 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705385+0000", 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705385+0000", 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705386+0000", 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0809999999999999e-06 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705923+0000", 2024-09-17T17:29:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000536949 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705968+0000", 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4969999999999998e-05 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705979+0000", 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0502e-05 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706074+0000", 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4996999999999999e-05 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706090+0000", 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5948999999999999e-05 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707076+0000", 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098629800000000004 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707105+0000", 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.915e-05 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/35 ECSubWrite(tid=13, reqid=client.4566.0:204, at_version=39'10, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.707190+0000", 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.82340378, 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091123899999999999, 2024-09-17T17:29:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707190+0000", 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707190+0000", 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707191+0000", 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0999999999999998e-07 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707331+0000", 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013977799999999999 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707355+0000", 2024-09-17T17:29:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4363999999999999e-05 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707361+0000", 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.429e-06 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707408+0000", 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7749e-05 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707416+0000", 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2760000000000003e-06 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.708050+0000", 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063450400000000004 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.708102+0000", 2024-09-17T17:29:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1529000000000001e-05 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 39/35 ECSubWrite(tid=27, reqid=client.4566.0:209, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8195151379999999, 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011625979999999999, 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711105+0000", 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5629000000000001e-05 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711164+0000", 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8548000000000003e-05 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711174+0000", 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0292000000000001e-05 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711311+0000", 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137324 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711321+0000", 2024-09-17T17:29:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0132000000000001e-05 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712206+0000", 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00088499800000000001 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712242+0000", 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5395000000000002e-05 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 39/35 ECSubWrite(tid=29, reqid=client.4566.0:210, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.712246+0000", 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8183482340000001, 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00079367999999999995, 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712246+0000", 2024-09-17T17:29:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712246+0000", 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712247+0000", 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712279+0000", 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1755999999999999e-05 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712336+0000", 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6861000000000003e-05 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712344+0000", 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2390000000000002e-06 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712403+0000", 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8770999999999997e-05 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712412+0000", 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5810000000000003e-06 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713012+0000", 2024-09-17T17:29:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059969699999999995 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713040+0000", 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7854000000000001e-05 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 39/35 ECSubWrite(tid=31, reqid=client.4566.0:211, at_version=39'8, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.713601+0000", 2024-09-17T17:29:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.816993342, 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082603799999999997, 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713601+0000", 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713601+0000", 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713602+0000", 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-07 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713620+0000", 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7883000000000001e-05 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713657+0000", 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6976e-05 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713665+0000", 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2889999999999998e-06 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713748+0000", 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3201999999999994e-05 2024-09-17T17:29:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713759+0000", 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.079e-05 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714390+0000", 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063071399999999999 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714427+0000", 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7299000000000003e-05 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 39/32 ECSubWrite(tid=34, reqid=client.4566.0:207, at_version=39'12, trim_to=0'0, roll_forward_to=36'10))", 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.714563+0000", 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8160313559999999, 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014888449999999999, 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714563+0000", 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714563+0000", 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714564+0000", 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714823+0000", 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025957399999999999 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714857+0000", 2024-09-17T17:29:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4078999999999997e-05 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714863+0000", 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6760000000000004e-06 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714929+0000", 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6049000000000004e-05 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714940+0000", 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1358999999999999e-05 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716004+0000", 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010639289999999999 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716052+0000", 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7555000000000001e-05 2024-09-17T17:29:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 39/32 ECSubWrite(tid=36, reqid=client.4566.0:207, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.717700+0000", 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8128942000000001, 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00085479100000000004, 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717700+0000", 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717700+0000", 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717701+0000", 2024-09-17T17:29:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7199999999999998e-07 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717713+0000", 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2169e-05 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717746+0000", 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2885000000000001e-05 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717754+0000", 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.619e-06 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717817+0000", 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2843999999999997e-05 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717826+0000", 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0450000000000006e-06 2024-09-17T17:29:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.718515+0000", 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068855999999999995 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.718555+0000", 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9897000000000001e-05 2024-09-17T17:29:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8100566900000001, 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016256700000000001, 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720975+0000", 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043686899999999999 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721002+0000", 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7030999999999999e-05 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721008+0000", 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.942e-06 2024-09-17T17:29:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721060+0000", 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1916000000000002e-05 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721066+0000", 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722130+0000", 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010640070000000001 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722163+0000", 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3198999999999998e-05 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=19, reqid=client.4566.0:217, at_version=39'15, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.722254+0000", 2024-09-17T17:29:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8083401029999999, 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010264950000000001, 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722254+0000", 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722254+0000", 2024-09-17T17:29:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722255+0000", 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4000000000000001e-07 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722596+0000", 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034096199999999999 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722632+0000", 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6186000000000002e-05 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722642+0000", 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6630000000000005e-06 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722695+0000", 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3421999999999998e-05 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722701+0000", 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5010000000000004e-06 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.723254+0000", 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055329100000000003 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.723281+0000", 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.673e-05 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:54.638 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:54.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:29:54.640 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:54.637+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:osd set-full-ratio 0.95 2024-09-17T17:29:54.694 INFO:tasks.rados.rados.0.smithi102.stdout:107: done (2 left) 2024-09-17T17:29:54.694 INFO:tasks.rados.rados.0.smithi102.stdout:108: delete oid 11 current snap is 4 2024-09-17T17:29:54.694 INFO:tasks.rados.rados.0.smithi102.stdout:105: expect (ObjNum 57 snap 2 seq_num 57) 2024-09-17T17:29:54.694 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:54.713 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:54.729 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:54.740 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:54.738+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:54.744 INFO:tasks.rados.rados.0.smithi102.stdout:100: finishing write tid 3 to smithi10238019-23 2024-09-17T17:29:54.744 INFO:tasks.rados.rados.0.smithi102.stdout:100: finishing write tid 4 to smithi10238019-23 2024-09-17T17:29:54.745 INFO:tasks.rados.rados.0.smithi102.stdout:100: oid 23 updating version 0 to 25 2024-09-17T17:29:54.745 INFO:tasks.rados.rados.0.smithi102.stdout:100: oid 23 version 25 is already newer than 24 2024-09-17T17:29:54.745 INFO:tasks.rados.rados.0.smithi102.stdout:100: oid 23 version 25 is already newer than 23 2024-09-17T17:29:54.745 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 23 v 25 (ObjNum 70 snap 4 seq_num 70) dirty exists 2024-09-17T17:29:54.745 INFO:tasks.rados.rados.0.smithi102.stdout:100: left oid 23 (ObjNum 70 snap 4 seq_num 70) 2024-09-17T17:29:54.747 INFO:tasks.rados.rados.0.smithi102.stdout:100: done (2 left) 2024-09-17T17:29:54.747 INFO:tasks.rados.rados.0.smithi102.stdout:105: done (1 left) 2024-09-17T17:29:54.747 INFO:tasks.rados.rados.0.smithi102.stdout:108: done (0 left) 2024-09-17T17:29:54.747 INFO:tasks.rados.rados.0.smithi102.stdout:109: read oid 33 snap -1 2024-09-17T17:29:54.747 INFO:tasks.rados.rados.0.smithi102.stdout:109: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-17T17:29:54.747 INFO:tasks.rados.rados.0.smithi102.stdout:110: delete oid 5 current snap is 4 2024-09-17T17:29:54.757 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:54.757 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:54.757 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:29:54.757 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:54.757 INFO:tasks.rados.rados.0.smithi102.stdout:109: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-17T17:29:54.758 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:54.762 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:54.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:29:54.767 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:54.841 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:54.839+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:54.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:29:54.942 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:54.941+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:54.963 INFO:tasks.rados.rados.0.smithi102.stdout:109: done (1 left) 2024-09-17T17:29:54.963 INFO:tasks.rados.rados.0.smithi102.stdout:110: done (0 left) 2024-09-17T17:29:54.963 INFO:tasks.rados.rados.0.smithi102.stdout:111: rmattr oid 6 current snap is 4 2024-09-17T17:29:54.963 INFO:tasks.rados.rados.0.smithi102.stdout:111: done (0 left) 2024-09-17T17:29:54.963 INFO:tasks.rados.rados.0.smithi102.stdout:112: rmattr oid 14 current snap is 4 2024-09-17T17:29:54.964 INFO:tasks.rados.rados.0.smithi102.stdout:112: done (0 left) 2024-09-17T17:29:54.964 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 6 current snap is 4 2024-09-17T17:29:54.964 INFO:tasks.rados.rados.0.smithi102.stdout:113: seq_num 75 ranges {98304=491520,589824=770048} 2024-09-17T17:29:54.966 INFO:tasks.rados.rados.0.smithi102.stdout:113: writing smithi10238019-6 from 98304 to 589824 tid 1 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:113: writing smithi10238019-6 from 589824 to 1359872 tid 2 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:114: rollback oid 39 current snap is 4 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 39 to 3 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:115: read oid 37 snap 3 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:115: expect (ObjNum 67 snap 3 seq_num 67) 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 16 current snap is 4 2024-09-17T17:29:54.971 INFO:tasks.rados.rados.0.smithi102.stdout:116: seq_num 76 ranges {1409024=507904,1916928=507904,2424832=262144} 2024-09-17T17:29:54.982 INFO:tasks.rados.rados.0.smithi102.stdout:116: writing smithi10238019-16 from 1409024 to 1916928 tid 1 2024-09-17T17:29:54.985 INFO:tasks.rados.rados.0.smithi102.stdout:116: writing smithi10238019-16 from 1916928 to 2424832 tid 2 2024-09-17T17:29:54.987 INFO:tasks.rados.rados.0.smithi102.stdout:116: writing smithi10238019-16 from 2424832 to 2686976 tid 3 2024-09-17T17:29:54.987 INFO:tasks.rados.rados.0.smithi102.stdout:117: rollback oid 30 current snap is 4 2024-09-17T17:29:54.987 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 30 to 3 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: finishing write tid 1 to smithi10238019-6 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:114: finishing rollback tid 0 to smithi10238019-39 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: finishing write tid 2 to smithi10238019-6 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:114: finishing rollback tid 1 to smithi10238019-39 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:114: finishing rollback tid 2 to smithi10238019-39 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 39 v 14 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: finishing write tid 3 to smithi10238019-6 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: finishing write tid 4 to smithi10238019-6 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: oid 6 updating version 0 to 21 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: oid 6 version 21 is already newer than 20 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: oid 6 updating version 21 to 22 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 6 v 22 (ObjNum 75 snap 4 seq_num 75) dirty exists 2024-09-17T17:29:54.988 INFO:tasks.rados.rados.0.smithi102.stdout:113: left oid 6 (ObjNum 75 snap 4 seq_num 75) 2024-09-17T17:29:54.989 INFO:tasks.rados.rados.0.smithi102.stdout:113: done (4 left) 2024-09-17T17:29:54.989 INFO:tasks.rados.rados.0.smithi102.stdout:114: done (3 left) 2024-09-17T17:29:54.989 INFO:tasks.rados.rados.0.smithi102.stdout:118: snap_create 2024-09-17T17:29:54.990 INFO:tasks.rados.rados.0.smithi102.stdout:117: finishing rollback tid 0 to smithi10238019-30 2024-09-17T17:29:54.990 INFO:tasks.rados.rados.0.smithi102.stdout:117: finishing rollback tid 1 to smithi10238019-30 2024-09-17T17:29:54.992 INFO:tasks.rados.rados.0.smithi102.stdout:117: finishing rollback tid 2 to smithi10238019-30 2024-09-17T17:29:54.992 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 30 v 24 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-17T17:29:54.993 INFO:tasks.rados.rados.0.smithi102.stdout:116: finishing write tid 1 to smithi10238019-16 2024-09-17T17:29:54.994 INFO:tasks.rados.rados.0.smithi102.stdout:116: finishing write tid 2 to smithi10238019-16 2024-09-17T17:29:54.994 INFO:tasks.rados.rados.0.smithi102.stdout:116: finishing write tid 3 to smithi10238019-16 2024-09-17T17:29:55.031 INFO:tasks.rados.rados.0.smithi102.stdout:116: finishing write tid 4 to smithi10238019-16 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:116: finishing write tid 5 to smithi10238019-16 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:116: oid 16 updating version 0 to 12 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:116: oid 16 updating version 12 to 13 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:116: oid 16 version 13 is already newer than 11 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:116: oid 16 updating version 13 to 14 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 14 (ObjNum 76 snap 4 seq_num 76) dirty exists 2024-09-17T17:29:55.032 INFO:tasks.rados.rados.0.smithi102.stdout:116: left oid 16 (ObjNum 76 snap 4 seq_num 76) 2024-09-17T17:29:55.036 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:55.036 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:29:55.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:29:55.036 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:48.991744+0000", 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.04233025, 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010000600000000001, 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991744+0000", 2024-09-17T17:29:55.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991744+0000", 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991745+0000", 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991777+0000", 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1730000000000003e-05 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991815+0000", 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8797000000000001e-05 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991824+0000", 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1829999999999994e-06 2024-09-17T17:29:55.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991891+0000", 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7619000000000004e-05 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.991902+0000", 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.039e-05 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.992714+0000", 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081247000000000003 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:48.992744+0000", 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0103e-05 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:49.069467+0000", 2024-09-17T17:29:55.041 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9646079140000001, 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086831199999999999, 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069467+0000", 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069467+0000", 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069468+0000", 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.66e-06 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069491+0000", 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3039000000000001e-05 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069559+0000", 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8026000000000002e-05 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069583+0000", 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.406e-05 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069726+0000", 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014224100000000001 2024-09-17T17:29:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.069734+0000", 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.67e-06 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.070306+0000", 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057205000000000005 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:49.070335+0000", 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8566000000000001e-05 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.349417882, 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:29:55.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=19, reqid=osd.15.0:19, at_version=39'15, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3457433480000001, 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001026447, 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7700000000000001e-07 2024-09-17T17:29:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688373+0000", 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2141999999999997e-05 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688428+0000", 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4759999999999997e-05 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688437+0000", 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5699999999999993e-06 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688514+0000", 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6958000000000004e-05 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688524+0000", 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-17T17:29:55.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.689315+0000", 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00079100500000000001 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.689358+0000", 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2840000000000003e-05 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=20, reqid=client.4566.0:191, at_version=39'17, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.343883548, 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001787521, 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:29:55.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690192+0000", 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690618+0000", 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042645600000000002 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690656+0000", 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8214999999999999e-05 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690666+0000", 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5419999999999994e-06 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690744+0000", 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8418999999999996e-05 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690754+0000", 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9329999999999996e-06 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691939+0000", 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001184198 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691978+0000", 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9845999999999998e-05 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=29, reqid=osd.3.0:28, at_version=0'0, trim_to=0'0, roll_forward_to=39'14))", 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.695033+0000", 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3390417130000003, 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000945359, 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695033+0000", 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695033+0000", 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695034+0000", 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.33e-06 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695053+0000", 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9032999999999999e-05 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695101+0000", 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7635e-05 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695111+0000", 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0750999999999999e-05 2024-09-17T17:29:55.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695198+0000", 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6488000000000003e-05 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695209+0000", 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1368e-05 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695937+0000", 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072746499999999995 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695978+0000", 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1288999999999997e-05 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.335881713, 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:29:55.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3336558539999999, 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:29:55.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:29:55.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3332317749999998, 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:29:55.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3331374729999999, 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:29:55.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3325125099999999, 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015435340000000001, 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701563+0000", 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.155e-06 2024-09-17T17:29:55.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701569+0000", 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701604+0000", 2024-09-17T17:29:55.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5148999999999998e-05 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701612+0000", 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9559999999999997e-06 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703084+0000", 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014716270000000001 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703105+0000", 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1766000000000001e-05 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3322752219999998, 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001340387, 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701800+0000", 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2900000000000002e-07 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701806+0000", 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5099999999999998e-06 2024-09-17T17:29:55.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701957+0000", 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000151175 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701965+0000", 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703127+0000", 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011621019999999999 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703140+0000", 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.295e-05 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.702013+0000", 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3320612670000003, 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001197989, 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702013+0000", 2024-09-17T17:29:55.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702013+0000", 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702014+0000", 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702021+0000", 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0269999999999998e-06 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702067+0000", 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6007999999999997e-05 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702075+0000", 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000007e-06 2024-09-17T17:29:55.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703188+0000", 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001112315 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703211+0000", 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3448e-05 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.702100+0000", 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.331974057, 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001135863, 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702100+0000", 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702100+0000", 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702101+0000", 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0800000000000003e-07 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702106+0000", 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0529999999999997e-06 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702144+0000", 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8058000000000001e-05 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702157+0000", 2024-09-17T17:29:55.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.289e-05 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703226+0000", 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010690949999999999 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703236+0000", 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.859e-06 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.7s1 39/32 ECSubWrite(tid=57, reqid=client.4566.0:182, at_version=39'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3306422490000003, 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00185337, 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:29:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703433+0000", 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4300000000000002e-07 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704005+0000", 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057213000000000001 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704034+0000", 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8662000000000001e-05 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704039+0000", 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1170000000000003e-06 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704101+0000", 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1818999999999999e-05 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704111+0000", 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0115000000000001e-05 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705248+0000", 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011371420000000001 2024-09-17T17:29:55.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705286+0000", 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7542000000000001e-05 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=23, reqid=client.4566.0:220, at_version=39'18, trim_to=0'0, roll_forward_to=39'17))", 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.721908+0000", 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3121665480000004, 2024-09-17T17:29:55.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087556299999999999, 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721908+0000", 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721908+0000", 2024-09-17T17:29:55.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721909+0000", 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4100000000000001e-06 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721936+0000", 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7083000000000001e-05 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721984+0000", 2024-09-17T17:29:55.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7750000000000002e-05 2024-09-17T17:29:55.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721995+0000", 2024-09-17T17:29:55.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0514e-05 2024-09-17T17:29:55.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722081+0000", 2024-09-17T17:29:55.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6139999999999999e-05 2024-09-17T17:29:55.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722089+0000", 2024-09-17T17:29:55.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2279999999999998e-06 2024-09-17T17:29:55.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722739+0000", 2024-09-17T17:29:55.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00065030399999999999 2024-09-17T17:29:55.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722783+0000", 2024-09-17T17:29:55.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4134000000000001e-05 2024-09-17T17:29:55.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=35, reqid=osd.3.0:35, at_version=43'18, trim_to=0'0, roll_forward_to=42'16))", 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:29:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.33668671300000003, 2024-09-17T17:29:55.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001758098, 2024-09-17T17:29:55.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:29:55.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:29:55.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:29:55.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:29:55.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697411+0000", 2024-09-17T17:29:55.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2937000000000001e-05 2024-09-17T17:29:55.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697449+0000", 2024-09-17T17:29:55.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519000000000002e-05 2024-09-17T17:29:55.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697458+0000", 2024-09-17T17:29:55.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-17T17:29:55.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697532+0000", 2024-09-17T17:29:55.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4147999999999995e-05 2024-09-17T17:29:55.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697542+0000", 2024-09-17T17:29:55.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.893e-06 2024-09-17T17:29:55.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699107+0000", 2024-09-17T17:29:55.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015656050000000001 2024-09-17T17:29:55.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699146+0000", 2024-09-17T17:29:55.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8525000000000001e-05 2024-09-17T17:29:55.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.132 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.bs4 43/32 ECSubWrite(tid=92, reqid=osd.13.0:92, at_version=43'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-17T17:29:55.132 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:29:55.132 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.336197782, 2024-09-17T17:29:55.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019485469999999999, 2024-09-17T17:29:55.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.133 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:29:55.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:29:55.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:29:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:29:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697917+0000", 2024-09-17T17:29:55.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9153999999999998e-05 2024-09-17T17:29:55.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697948+0000", 2024-09-17T17:29:55.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1538000000000002e-05 2024-09-17T17:29:55.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697957+0000", 2024-09-17T17:29:55.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:29:55.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698771+0000", 2024-09-17T17:29:55.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081437600000000001 2024-09-17T17:29:55.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698781+0000", 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9590000000000007e-06 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699776+0000", 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099482900000000003 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699825+0000", 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9280000000000003e-05 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.bs4 43/32 ECSubWrite(tid=93, reqid=osd.13.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=43'8))", 2024-09-17T17:29:55.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.705922+0000", 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.32815238099999999, 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00092373200000000003, 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.705922+0000", 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.705922+0000", 2024-09-17T17:29:55.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.705923+0000", 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.079e-06 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.705937+0000", 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3915e-05 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.705986+0000", 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9403000000000001e-05 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.705997+0000", 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0241000000000001e-05 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706072+0000", 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5333999999999994e-05 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706086+0000", 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3879e-05 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706805+0000", 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071858700000000002 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706846+0000", 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1294000000000001e-05 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=36, reqid=osd.3.0:35, at_version=0'0, trim_to=0'0, roll_forward_to=43'18))", 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.706744+0000", 2024-09-17T17:29:55.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.32733009099999999, 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089621899999999997, 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706744+0000", 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706744+0000", 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706745+0000", 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.47e-07 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706759+0000", 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414000000000001e-05 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706791+0000", 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1980000000000002e-05 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706806+0000", 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5146e-05 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706870+0000", 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3961999999999995e-05 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706879+0000", 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5079999999999994e-06 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707601+0000", 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072171799999999999 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707641+0000", 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9644000000000002e-05 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.156 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:55.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:29:55.158 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:55.155+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:55.258 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:55.256+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:55.266 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:29:55.266 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:55.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:29:55.359 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:55.357+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:55.381 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:29:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:29:55.382 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:29:55.382 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:55.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:29:55.460 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:55.458+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:55.481 INFO:tasks.rados.rados.0.smithi102.stdout:115: expect (ObjNum 67 snap 3 seq_num 67) 2024-09-17T17:29:55.486 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1091472 2024-09-17T17:29:55.495 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 431326 2024-09-17T17:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8130734339999997, 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s1 39/35 ECSubWrite(tid=11, reqid=osd.1.0:10, at_version=0'0, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685357+0000", 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8107410809999998, 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018332299999999999, 2024-09-17T17:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685357+0000", 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685357+0000", 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685358+0000", 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.032e-06 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685371+0000", 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2689e-05 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685408+0000", 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6888000000000003e-05 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685418+0000", 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8560000000000002e-06 2024-09-17T17:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685672+0000", 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025456600000000002 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685683+0000", 2024-09-17T17:29:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0806e-05 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687154+0000", 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014711539999999999 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687191+0000", 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6239000000000003e-05 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s4 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8039480680000004, 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022843400000000002, 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:29:55.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-07 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692605+0000", 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045479800000000002 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692645+0000", 2024-09-17T17:29:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0033000000000001e-05 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692655+0000", 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0156999999999999e-05 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692715+0000", 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9502000000000001e-05 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692721+0000", 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.105e-06 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694369+0000", 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016482650000000001 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694435+0000", 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5369000000000004e-05 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s1 39/35 ECSubWrite(tid=12, reqid=client.4566.0:203, at_version=39'9, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.705448+0000", 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7906501820000003, 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00153115, 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705448+0000", 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705448+0000", 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705449+0000", 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6299999999999993e-07 2024-09-17T17:29:55.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705926+0000", 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047663600000000001 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705967+0000", 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1480999999999998e-05 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705979+0000", 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1228e-05 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706072+0000", 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3195999999999993e-05 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706083+0000", 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1504e-05 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706939+0000", 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00085557199999999997 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706979+0000", 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.057e-05 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:209 3.9s0 3:96cbaefd:::smithi10238019-46:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710441+0000", 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7856577060000003, 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064948369999999998, 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 209 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710441+0000", 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710441+0000", 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710442+0000", 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4619999999999999e-06 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710449+0000", 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710461+0000", 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2629000000000001e-05 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710467+0000", 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8560000000000003e-06 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710519+0000", 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1801000000000001e-05 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710596+0000", 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7019000000000002e-05 2024-09-17T17:29:55.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711059+0000", 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046334700000000002 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714258+0000", 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031989259999999999 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716853+0000", 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002594369 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716936+0000", 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2948000000000006e-05 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:210 3.9s0 3:96cbaefd:::smithi10238019-46:head [rollback 3] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7856138960000001, 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065835980000000004, 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 210 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710487+0000", 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8369999999999999e-06 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710492+0000", 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5860000000000004e-06 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710496+0000", 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3239999999999999e-06 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711515+0000", 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010192700000000001 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711606+0000", 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1471000000000006e-05 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712163+0000", 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055688199999999997 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714262+0000", 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020989889999999999 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717028+0000", 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027660889999999998 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717068+0000", 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9740999999999998e-05 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:211 3.9s0 3:96cbaefd:::smithi10238019-46:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7855845649999997, 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068972499999999997, 2024-09-17T17:29:55.532 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 211 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0400000000000002e-07 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710516+0000", 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5269999999999999e-06 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710520+0000", 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4059999999999998e-06 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710524+0000", 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.518e-06 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712807+0000", 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002283322 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713024+0000", 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216629 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713484+0000", 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046035200000000003 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714391+0000", 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090667600000000003 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717343+0000", 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002952863 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717411+0000", 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7652999999999998e-05 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:212 3.0s0 3:0b65e655:::smithi10238019-14:head [delete] snapc 3=[3] ondisk+write+known_if_redirected+full_force+supports_pool_eio e39)", 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.785569519, 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002126724, 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 212 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710530+0000", 2024-09-17T17:29:55.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1349999999999999e-06 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710538+0000", 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6930000000000007e-06 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710544+0000", 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0920000000000003e-06 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710600+0000", 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5315999999999997e-05 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710660+0000", 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0439000000000002e-05 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711125+0000", 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046455200000000002 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712320+0000", 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001195577 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712597+0000", 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027682199999999997 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712656+0000", 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8684000000000003e-05 2024-09-17T17:29:55.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 39/35 ECSubWriteReply(tid=27, last_complete=39'6, committed=1, applied=1))", 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.711934+0000", 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.78416481, 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017909989999999999, 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711934+0000", 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711934+0000", 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711935+0000", 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0500000000000002e-07 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711940+0000", 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1399999999999999e-06 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711973+0000", 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3061000000000001e-05 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711981+0000", 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4290000000000001e-06 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713707+0000", 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001725643 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713725+0000", 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7821000000000001e-05 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 39/35 ECSubWriteReply(tid=27, last_complete=39'6, committed=1, applied=1))", 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.712196+0000", 2024-09-17T17:29:55.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.78390206, 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015677589999999999, 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712196+0000", 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712196+0000", 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712197+0000", 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000005e-07 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712203+0000", 2024-09-17T17:29:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7350000000000001e-06 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712241+0000", 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7941999999999997e-05 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712250+0000", 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6789999999999994e-06 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713750+0000", 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001500417 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713764+0000", 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144e-05 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 39/35 ECSubWriteReply(tid=27, last_complete=39'6, committed=1, applied=1))", 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.712257+0000", 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7838417639999999, 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015650480000000001, 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712257+0000", 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712257+0000", 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712257+0000", 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712259+0000", 2024-09-17T17:29:55.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.632e-06 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712327+0000", 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8423000000000005e-05 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712333+0000", 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.054e-06 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713789+0000", 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014562819999999999 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713822+0000", 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2427e-05 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.556 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds3 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.80320830499999996, 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017461569999999999, 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.98e-07 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692920+0000", 2024-09-17T17:29:55.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9940000000000001e-05 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692998+0000", 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7701999999999996e-05 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693009+0000", 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1277999999999999e-05 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693076+0000", 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6647000000000001e-05 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693087+0000", 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0712e-05 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694596+0000", 2024-09-17T17:29:55.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015095600000000001 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694636+0000", 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0120000000000002e-05 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds3 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:29:55.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.693844+0000", 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.80225407800000004, 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025301949999999998, 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693844+0000", 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693844+0000", 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693845+0000", 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693860+0000", 2024-09-17T17:29:55.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4789e-05 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693885+0000", 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5157e-05 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0099999999999995e-06 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693969+0000", 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6512000000000002e-05 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693981+0000", 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.145e-05 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696340+0000", 2024-09-17T17:29:55.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002359566 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696375+0000", 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4076999999999999e-05 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.698102+0000", 2024-09-17T17:29:55.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.79799599799999998, 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015943719999999999, 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698102+0000", 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698102+0000", 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698103+0000", 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5300000000000001e-07 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698120+0000", 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7388e-05 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698188+0000", 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8238999999999995e-05 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698196+0000", 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698265+0000", 2024-09-17T17:29:55.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9519999999999998e-05 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698276+0000", 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0539e-05 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699671+0000", 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013955440000000001 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699697+0000", 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5508999999999999e-05 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 43/35 ECSubWrite(tid=54, reqid=osd.11.0:54, at_version=43'10, trim_to=0'0, roll_forward_to=42'8))", 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.79211388299999996, 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030555130000000002, 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.66e-07 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704004+0000", 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9400999999999999e-05 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706156+0000", 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021521719999999999 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706167+0000", 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0804e-05 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706242+0000", 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5234000000000005e-05 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706253+0000", 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0755e-05 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707001+0000", 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074821000000000002 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707040+0000", 2024-09-17T17:29:55.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8770999999999999e-05 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 43/32 ECSubWrite(tid=55, reqid=osd.11.0:55, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.79209313800000003, 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039008089999999999, 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6999999999999999e-07 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704014+0000", 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2509999999999994e-06 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706197+0000", 2024-09-17T17:29:55.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021834990000000002 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706204+0000", 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3439999999999997e-06 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706345+0000", 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000141488 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706355+0000", 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5489999999999995e-06 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707877+0000", 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001522175 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707906+0000", 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9332999999999999e-05 2024-09-17T17:29:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.78212021099999995, 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:29:55.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:29:55.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.74981856800000002, 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:29:55.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:29:55.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:29:55.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:29:55.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:29:55.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.592 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:29:55.592 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:29:55.592 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.74789406599999997, 2024-09-17T17:29:55.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:29:55.592 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s1 43/32 ECSubWrite(tid=15, reqid=client.4566.0:307, at_version=43'12, trim_to=0'0, roll_forward_to=39'10))", 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52493224999999999, 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001720982, 2024-09-17T17:29:55.596 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:29:55.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971190+0000", 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-05 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:29:55.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971224+0000", 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3748999999999998e-05 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971231+0000", 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1339999999999998e-06 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971285+0000", 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971294+0000", 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6840000000000008e-06 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972838+0000", 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001543981 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972887+0000", 2024-09-17T17:29:55.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9013e-05 2024-09-17T17:29:55.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:29:55.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:29:55.601 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:29:55.602 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:55.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:29:55.603 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:55.600+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:55.694 INFO:tasks.rados.rados.0.smithi102.stdout:115: done (3 left) 2024-09-17T17:29:55.694 INFO:tasks.rados.rados.0.smithi102.stdout:116: done (2 left) 2024-09-17T17:29:55.694 INFO:tasks.rados.rados.0.smithi102.stdout:117: done (1 left) 2024-09-17T17:29:55.694 INFO:tasks.rados.rados.0.smithi102.stdout:118: done (0 left) 2024-09-17T17:29:55.694 INFO:tasks.rados.rados.0.smithi102.stdout:119: snap_create 2024-09-17T17:29:55.703 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:55.701+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:55.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:29:55.725 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:55.725 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:55.725 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:29:55.725 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:55.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:29:55.804 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:55.802+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:55.843 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:55.843 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:55.843 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:29:55.843 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:29:55.843 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:55.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:29:55.905 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:55.902+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:56.000 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:29:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:29:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.5209349750000003, 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:29:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:179 3.4s0 3:26d49efd:::smithi10238019-44:head [delete] snapc 2=[2] ondisk+write+known_if_redirected+full_force+supports_pool_eio e37)", 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.0075403620000003, 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039941960000000002, 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:29:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 179 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990179+0000", 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.22e-06 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990184+0000", 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1239999999999998e-06 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990194+0000", 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0377000000000001e-05 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990200+0000", 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5690000000000001e-06 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990248+0000", 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7615999999999999e-05 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990300+0000", 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2364000000000002e-05 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991074+0000", 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077367199999999999 2024-09-17T17:29:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992851+0000", 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001776632 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994099+0000", 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012486610000000001 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994172+0000", 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2960999999999995e-05 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.0026687369999996, 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:29:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:29:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:29:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s2 39/35 ECSubWrite(tid=14, reqid=client.4566.0:184, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.678477+0000", 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.3192420609999997, 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027762559999999999, 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678477+0000", 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678477+0000", 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678477+0000", 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7599999999999998e-07 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678494+0000", 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7133e-05 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679450+0000", 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095585400000000001 2024-09-17T17:29:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679460+0000", 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0284e-05 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679526+0000", 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6289000000000002e-05 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679536+0000", 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0118999999999999e-05 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681226+0000", 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0016897119999999999 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681253+0000", 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6588999999999999e-05 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s2 39/35 ECSubWrite(tid=15, reqid=client.4566.0:185, at_version=39'11, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.679414+0000", 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.3183047200000004, 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023665800000000001, 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679414+0000", 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679414+0000", 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679415+0000", 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9400000000000005e-07 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679433+0000", 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8119999999999999e-05 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680604+0000", 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001170934 2024-09-17T17:29:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680612+0000", 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0369999999999995e-06 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680681+0000", 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9383999999999998e-05 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680691+0000", 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.025e-05 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681743+0000", 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001051452 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681780+0000", 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7709000000000001e-05 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s2 39/35 ECSubWrite(tid=16, reqid=client.4566.0:186, at_version=39'12, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.680570+0000", 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.3171489190000001, 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033317799999999999, 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680570+0000", 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680570+0000", 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680570+0000", 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4000000000000001e-07 2024-09-17T17:29:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680587+0000", 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6252999999999998e-05 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683056+0000", 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024692429999999999 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683071+0000", 2024-09-17T17:29:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5594999999999999e-05 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683125+0000", 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.397e-05 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683135+0000", 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4700000000000008e-06 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683868+0000", 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00073326299999999995 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683901+0000", 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3346000000000002e-05 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=10, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.683033+0000", 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.3146856439999999, 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002429133, 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.032 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683033+0000", 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683033+0000", 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683034+0000", 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1200000000000002e-07 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683037+0000", 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3230000000000002e-06 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684792+0000", 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017549340000000001 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.034 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684800+0000", 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6369999999999999e-06 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684849+0000", 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9839000000000002e-05 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.685462+0000", 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00061268800000000001 2024-09-17T17:29:56.035 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s2 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.684765+0000", 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.3129538600000004, 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028185530000000001, 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.036 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684765+0000", 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684765+0000", 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.037 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684765+0000", 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2500000000000005e-07 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684775+0000", 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0006999999999999e-05 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686797+0000", 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002021274 2024-09-17T17:29:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686805+0000", 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2260000000000005e-06 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686871+0000", 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6081000000000006e-05 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686881+0000", 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7680000000000003e-06 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.687558+0000", 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000677091 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.687583+0000", 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5480999999999999e-05 2024-09-17T17:29:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.3109512140000001, 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:29:56.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.043 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:29:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:203 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~557056] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2937503030000004, 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0069058180000000002, 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:29:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 203 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703971+0000", 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4849999999999999e-06 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704582+0000", 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00061129700000000001 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.047 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704592+0000", 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0094e-05 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704598+0000", 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6019999999999999e-06 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704655+0000", 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7107000000000001e-05 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704712+0000", 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7435000000000002e-05 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.705382+0000", 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067020899999999995 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706343+0000", 2024-09-17T17:29:56.049 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00096062299999999997 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710818+0000", 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044745519999999997 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710874+0000", 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6413999999999999e-05 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.291507932, 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:56.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.053 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:29:56.055 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2911210290000001, 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:56.057 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:29:56.059 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:29:56.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2910876619999998, 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:29:56.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.065 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.066 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:29:56.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.068 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=12, last_complete=39'9, committed=1, applied=1))", 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.708236+0000", 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2894829000000003, 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026464700000000002, 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.069 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708236+0000", 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708236+0000", 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708236+0000", 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.75e-07 2024-09-17T17:29:56.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708242+0000", 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3510000000000001e-06 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710743+0000", 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002501035 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.071 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710748+0000", 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.293e-06 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710774+0000", 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6347e-05 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710882+0000", 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010786899999999999 2024-09-17T17:29:56.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2866381909999998, 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:29:56.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:29:56.076 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s6 39/35 ECSubWrite(tid=27, reqid=client.4566.0:209, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.713524+0000", 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2841944410000004, 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0031328250000000001, 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713524+0000", 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713524+0000", 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713525+0000", 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713545+0000", 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0412999999999999e-05 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715742+0000", 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021970280000000002 2024-09-17T17:29:56.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715751+0000", 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6179999999999995e-06 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715807+0000", 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6662000000000001e-05 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715818+0000", 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0427e-05 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716639+0000", 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000820676 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716657+0000", 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8445999999999999e-05 2024-09-17T17:29:56.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s6 39/35 ECSubWrite(tid=29, reqid=client.4566.0:210, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.713548+0000", 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2841702719999999, 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033443930000000002, 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713548+0000", 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713548+0000", 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713549+0000", 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0899999999999997e-07 2024-09-17T17:29:56.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713566+0000", 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7529000000000001e-05 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715774+0000", 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002207741 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715780+0000", 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9699999999999996e-06 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716322+0000", 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054210100000000002 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716326+0000", 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3229999999999999e-06 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716874+0000", 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054769900000000002 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716893+0000", 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8720999999999999e-05 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s6 39/35 ECSubWrite(tid=31, reqid=client.4566.0:211, at_version=39'8, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.713568+0000", 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2841501720000004, 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003603948, 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713568+0000", 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713568+0000", 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713569+0000", 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0899999999999997e-07 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.088 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713578+0000", 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8419999999999994e-06 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715796+0000", 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022186279999999998 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715801+0000", 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8289999999999998e-06 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716515+0000", 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000713676 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716520+0000", 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2020000000000003e-06 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.717153+0000", 2024-09-17T17:29:56.090 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00063337699999999996 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.717172+0000", 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9085000000000001e-05 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s2 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.720566+0000", 2024-09-17T17:29:56.091 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.2771527799999998, 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028424399999999999, 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720566+0000", 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720566+0000", 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720566+0000", 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4499999999999997e-07 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720946+0000", 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037978900000000001 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722040+0000", 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001093994 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722048+0000", 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3649999999999998e-06 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722105+0000", 2024-09-17T17:29:56.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7417999999999998e-05 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722113+0000", 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.892e-06 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723372+0000", 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012590539999999999 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.095 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723408+0000", 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6282999999999998e-05 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s6 43/35 ECSubWrite(tid=44, reqid=osd.12.0:43, at_version=0'0, trim_to=0'0, roll_forward_to=43'17))", 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.710913+0000", 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.2868054440000001, 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003030399, 2024-09-17T17:29:56.096 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.710913+0000", 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.097 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.710913+0000", 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.710913+0000", 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.710919+0000", 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0719999999999996e-06 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.713095+0000", 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002175349 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.713100+0000", 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5470000000000004e-06 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.713227+0000", 2024-09-17T17:29:56.099 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012636799999999999 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.713240+0000", 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3682e-05 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.713901+0000", 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00066107899999999996 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.100 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.713944+0000", 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2069e-05 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.101 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:56.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:29:56.103 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:56.101+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:56.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:29:56.203 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:56.201+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:56.210 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:56.210 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:56.210 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:29:56.210 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:56.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:29:56.291 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:56.291 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:56.305 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:56.303+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:56.313 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:56.313 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:56.326 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:56.326 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:29:56.326 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:29:56.326 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:29:56.326 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:56.335 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:29:56.420 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:56.418+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:56.484 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.4904640469999997, 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.485 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:29:56.486 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.487 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:29:56.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=16, reqid=client.4566.0:165, at_version=37'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.4881717590000001, 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0014151649999999999, 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994059+0000", 2024-09-17T17:29:56.491 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7000000000000004e-07 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994076+0000", 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7873999999999999e-05 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994174+0000", 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7345999999999999e-05 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994183+0000", 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7739999999999993e-06 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994549+0000", 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00036644599999999998 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994559+0000", 2024-09-17T17:29:56.493 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0019e-05 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995447+0000", 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00088779500000000003 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995473+0000", 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6241000000000002e-05 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=17, reqid=client.4566.0:166, at_version=37'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.4881497469999996, 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001407352, 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.496 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994080+0000", 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6699999999999999e-07 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994094+0000", 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3264e-05 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994201+0000", 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107019 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994206+0000", 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4519999999999996e-06 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994836+0000", 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000629709 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994844+0000", 2024-09-17T17:29:56.498 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.841e-06 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995476+0000", 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00063211799999999998 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995487+0000", 2024-09-17T17:29:56.499 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1582e-05 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=14, reqid=client.4566.0:184, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.678553+0000", 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.8036770000000004, 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001206373, 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678553+0000", 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678553+0000", 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678553+0000", 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7500000000000001e-07 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678571+0000", 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8119999999999999e-05 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678613+0000", 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2107000000000002e-05 2024-09-17T17:29:56.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678622+0000", 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8289999999999997e-06 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678697+0000", 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4537999999999997e-05 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.503 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678710+0000", 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.295e-05 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679737+0000", 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010271600000000001 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679759+0000", 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2294e-05 2024-09-17T17:29:56.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=15, reqid=client.4566.0:185, at_version=39'11, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.679451+0000", 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.8027788969999996, 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001394129, 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679451+0000", 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679451+0000", 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679451+0000", 2024-09-17T17:29:56.506 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1500000000000004e-07 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679466+0000", 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4025999999999999e-05 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679495+0000", 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9266000000000001e-05 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.507 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679503+0000", 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0290000000000005e-06 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679556+0000", 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3000000000000001e-05 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.679562+0000", 2024-09-17T17:29:56.508 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9139999999999996e-06 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680812+0000", 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012505909999999999 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.680845+0000", 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2588e-05 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.684789+0000", 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.7974411269999999, 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015238630000000001, 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684789+0000", 2024-09-17T17:29:56.510 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684789+0000", 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684790+0000", 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7399999999999991e-07 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.511 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684817+0000", 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7800000000000001e-05 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684856+0000", 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8268e-05 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684874+0000", 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8845e-05 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.512 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.685099+0000", 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000224434 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8009999999999998e-06 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686110+0000", 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001001395 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686312+0000", 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000202346 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 39/35 ECSubWrite(tid=20, reqid=client.4566.0:191, at_version=39'17, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.689581+0000", 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.792648571, 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001530132, 2024-09-17T17:29:56.514 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689581+0000", 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689581+0000", 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.515 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689582+0000", 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6700000000000003e-07 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689753+0000", 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017074899999999999 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689784+0000", 2024-09-17T17:29:56.516 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1124000000000002e-05 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689790+0000", 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7959999999999998e-06 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689845+0000", 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4627999999999999e-05 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.517 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689856+0000", 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0678e-05 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.691077+0000", 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012208570000000001 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.691111+0000", 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4632999999999999e-05 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.518 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 39/32 ECSubWrite(tid=15, reqid=client.4566.0:199, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.700290+0000", 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.7819398199999998, 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001229045, 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.519 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700290+0000", 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700290+0000", 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700290+0000", 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.53e-07 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.520 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00025112300000000001 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700566+0000", 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4281999999999999e-05 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700571+0000", 2024-09-17T17:29:56.521 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.57e-06 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700631+0000", 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9410999999999998e-05 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.700641+0000", 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0118999999999999e-05 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.522 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.701488+0000", 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00084777699999999997 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.701519+0000", 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0409999999999999e-05 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:56.523 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.7615277420000002, 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002528565, 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.524 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720703+0000", 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5300000000000004e-07 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721318+0000", 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000615779 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.525 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721351+0000", 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2486e-05 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721360+0000", 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8729999999999996e-06 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721434+0000", 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4011999999999995e-05 2024-09-17T17:29:56.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721445+0000", 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0903999999999999e-05 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723198+0000", 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001753641 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723231+0000", 2024-09-17T17:29:56.527 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2217e-05 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 39/35 ECSubWrite(tid=23, reqid=client.4566.0:220, at_version=39'18, trim_to=0'0, roll_forward_to=39'17))", 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.722057+0000", 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.7601723199999997, 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00151669, 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.528 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722057+0000", 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722057+0000", 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.529 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722058+0000", 2024-09-17T17:29:56.530 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0599999999999999e-07 2024-09-17T17:29:56.530 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.530 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.530 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.530 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722107+0000", 2024-09-17T17:29:56.530 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9092999999999999e-05 2024-09-17T17:29:56.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.531 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722141+0000", 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3643000000000003e-05 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722149+0000", 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9459999999999998e-06 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722259+0000", 2024-09-17T17:29:56.532 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010987 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722269+0000", 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0155e-05 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723541+0000", 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001272256 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.533 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723574+0000", 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2920999999999999e-05 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=19, reqid=client.4566.0:217, at_version=39'15, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.722417+0000", 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.7598123220000002, 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001461855, 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.534 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722417+0000", 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722417+0000", 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722418+0000", 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9800000000000003e-07 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722672+0000", 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00025434600000000002 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722713+0000", 2024-09-17T17:29:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0831000000000003e-05 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722721+0000", 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0579999999999998e-06 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722778+0000", 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6549999999999999e-05 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722788+0000", 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8109999999999997e-06 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723840+0000", 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001052362 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723879+0000", 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9199000000000002e-05 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.7563910069999999, 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:29:56.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:29:56.541 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.542 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.543 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.692888+0000", 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.789341294, 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001928377, 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692888+0000", 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.544 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692888+0000", 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692889+0000", 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5200000000000002e-07 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692945+0000", 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5787e-05 2024-09-17T17:29:56.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692999+0000", 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4858999999999998e-05 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693011+0000", 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1496e-05 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.546 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693138+0000", 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000126744 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693148+0000", 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0583e-05 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694769+0000", 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00162119 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.547 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694817+0000", 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7265999999999998e-05 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=37, reqid=osd.8.0:37, at_version=43'27, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.7891515170000001, 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022426339999999999, 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.549 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9400000000000001e-07 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693102+0000", 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3547e-05 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693145+0000", 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3263000000000002e-05 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693153+0000", 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6690000000000006e-06 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693240+0000", 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7158999999999997e-05 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693250+0000", 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695273+0000", 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020229200000000001 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695321+0000", 2024-09-17T17:29:56.552 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8044000000000003e-05 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.788363135, 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:29:56.555 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.556 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.557 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.7881210250000001, 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.558 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.559 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.560 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.700980+0000", 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.781249227, 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020660069999999999, 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700980+0000", 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700980+0000", 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700981+0000", 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1399999999999996e-07 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700995+0000", 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3764000000000001e-05 2024-09-17T17:29:56.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701031+0000", 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5837000000000003e-05 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701040+0000", 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.565 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701116+0000", 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6530000000000001e-05 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701126+0000", 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5990000000000008e-06 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703000+0000", 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001874173 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.566 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703046+0000", 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6338999999999998e-05 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.701573+0000", 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.780656939, 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015107689999999999, 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701573+0000", 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701573+0000", 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.568 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701574+0000", 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1100000000000004e-07 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701591+0000", 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6962000000000001e-05 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701693+0000", 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010230000000000001 2024-09-17T17:29:56.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701702+0000", 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2869999999999993e-06 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701812+0000", 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010932 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701822+0000", 2024-09-17T17:29:56.570 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0628000000000001e-05 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703037+0000", 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001214726 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703084+0000", 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6635000000000003e-05 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 43/32 ECSubWrite(tid=55, reqid=osd.11.0:55, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.7805977579999999, 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002034059, 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.572 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.573 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701655+0000", 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2809000000000001e-05 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701737+0000", 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2349999999999996e-05 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701745+0000", 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701985+0000", 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023929299999999999 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701996+0000", 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.118e-05 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.575 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703604+0000", 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001608682 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703666+0000", 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1535e-05 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 43/32 ECSubWrite(tid=56, reqid=osd.11.0:55, at_version=0'0, trim_to=0'0, roll_forward_to=43'21))", 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.714403+0000", 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.767827128, 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011985330000000001, 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714403+0000", 2024-09-17T17:29:56.577 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714403+0000", 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714404+0000", 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6449999999999999e-06 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:29:56.578 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714425+0000", 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0421999999999999e-05 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714476+0000", 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1107999999999999e-05 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714496+0000", 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0667999999999999e-05 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.579 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714603+0000", 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010704000000000001 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.714614+0000", 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1014e-05 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.715566+0000", 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095114899999999998 2024-09-17T17:29:56.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.715601+0000", 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5487000000000001e-05 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:29:56.581 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:29:56.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:56.583 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:56.583 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:56.583 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:56.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:29:56.584 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:29:56.585 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:29:56.585 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:56.580+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:56.601 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:29:56.601 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:29:56.622 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:29:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:29:56.646 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:29:56.647 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:29:56.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:29:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:29:56.683 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:56.681+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:56.687 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:29:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:29:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:29:56.699 INFO:tasks.rados.rados.0.smithi102.stdout:119: done (0 left) 2024-09-17T17:29:56.699 INFO:tasks.rados.rados.0.smithi102.stdout:120: read oid 24 snap 3 2024-09-17T17:29:56.699 INFO:tasks.rados.rados.0.smithi102.stdout:120: expect (ObjNum 65 snap 3 seq_num 65) 2024-09-17T17:29:56.700 INFO:tasks.rados.rados.0.smithi102.stdout:121: rollback oid 9 current snap is 6 2024-09-17T17:29:56.700 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 9 to 3 2024-09-17T17:29:56.700 INFO:tasks.rados.rados.0.smithi102.stdout:122: snap_create 2024-09-17T17:29:56.706 INFO:tasks.rados.rados.0.smithi102.stdout:121: finishing rollback tid 0 to smithi10238019-9 2024-09-17T17:29:56.707 INFO:tasks.rados.rados.0.smithi102.stdout:120: expect (ObjNum 65 snap 3 seq_num 65) 2024-09-17T17:29:56.707 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:56.713 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:29:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:29:56.729 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1313800 2024-09-17T17:29:56.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:29:56.736 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:29:56.737 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:56.737 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:56.737 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 757590 2024-09-17T17:29:56.742 INFO:tasks.rados.rados.0.smithi102.stdout:121: finishing rollback tid 1 to smithi10238019-9 2024-09-17T17:29:56.742 INFO:tasks.rados.rados.0.smithi102.stdout:121: finishing rollback tid 2 to smithi10238019-9 2024-09-17T17:29:56.742 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 18 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-17T17:29:56.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:29:56.756 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:29:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:29:56.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:29:56.777 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:29:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:29:56.780+0000 7fd956ffd640 0 -- 172.21.15.7:0/2485745771 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fd958221670 msgr2=0x7fd958223a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:29:56.791 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:56.787+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:56.853 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:56.853 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:56.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:29:56.885 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:56.883+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:56.986 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:56.984+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:57.009 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.009 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:29:57.086 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:57.085+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:57.124 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.125 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:29:57.187 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:57.185+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:57.241 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.241 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:29:57.288 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:29:57.286+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:29:57.389 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:57.387+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:57.398 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.399 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:29:57.489 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:57.488+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:57.527 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.527 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:29:57.591 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:57.589+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:57.644 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.644 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.653 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:29:57.696 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:29:57.694+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:29:57.701 INFO:tasks.rados.rados.0.smithi102.stdout:120: done (2 left) 2024-09-17T17:29:57.701 INFO:tasks.rados.rados.0.smithi102.stdout:121: done (1 left) 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:122: done (0 left) 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:123: copy_from oid 7 from oid 16 current snap is 7 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:124: read oid 20 snap -1 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:124: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:125: rmattr oid 8 current snap is 7 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:125: done (2 left) 2024-09-17T17:29:57.702 INFO:tasks.rados.rados.0.smithi102.stdout:126: snap_create 2024-09-17T17:29:57.709 INFO:tasks.rados.rados.0.smithi102.stdout:124: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-17T17:29:57.710 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 132981 2024-09-17T17:29:57.713 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 180953 2024-09-17T17:29:57.714 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:57.722 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:29:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:29:57.723 INFO:tasks.rados.rados.0.smithi102.stdout:123: finishing copy_from racing read to smithi10238019-7 2024-09-17T17:29:57.733 INFO:tasks.rados.rados.0.smithi102.stdout:123: finishing copy_from to smithi10238019-7 2024-09-17T17:29:57.733 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 14 (ObjNum 76 snap 4 seq_num 76) dirty exists 2024-09-17T17:29:57.747 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:29:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:29:57.762 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.762 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:29:57.792 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:57.791+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:57.893 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:57.891+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:57.919 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:57.919 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:57.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:29:57.994 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:57.992+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:58.074 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.074 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:29:58.095 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:58.094+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:58.189 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.189 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.196 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:29:58.194+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:29:58.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:29:58.297 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:29:58.295+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:29:58.306 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.306 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:29:58.398 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:58.395+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:58.421 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.421 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:29:58.498 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:58.496+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:58.536 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.537 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:29:58.599 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:29:58.597+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:29:58.652 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.652 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:29:58.701 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:58.699+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:58.709 INFO:tasks.rados.rados.0.smithi102.stdout:123: done (2 left) 2024-09-17T17:29:58.709 INFO:tasks.rados.rados.0.smithi102.stdout:124: done (1 left) 2024-09-17T17:29:58.709 INFO:tasks.rados.rados.0.smithi102.stdout:126: done (0 left) 2024-09-17T17:29:58.709 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 35 current snap is 8 2024-09-17T17:29:58.709 INFO:tasks.rados.rados.0.smithi102.stdout:127: seq_num 77 ranges {573440=524288,1097728=524288,1622016=245760} 2024-09-17T17:29:58.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:29:58.714 INFO:tasks.rados.rados.0.smithi102.stdout:127: writing smithi10238019-35 from 573440 to 1097728 tid 1 2024-09-17T17:29:58.718 INFO:tasks.rados.rados.0.smithi102.stdout:127: writing smithi10238019-35 from 1097728 to 1622016 tid 2 2024-09-17T17:29:58.720 INFO:tasks.rados.rados.0.smithi102.stdout:127: writing smithi10238019-35 from 1622016 to 1867776 tid 3 2024-09-17T17:29:58.721 INFO:tasks.rados.rados.0.smithi102.stdout:128: read oid 3 snap -1 2024-09-17T17:29:58.721 INFO:tasks.rados.rados.0.smithi102.stdout:128: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-17T17:29:58.721 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 10 current snap is 8 2024-09-17T17:29:58.721 INFO:tasks.rados.rados.0.smithi102.stdout:129: seq_num 78 ranges {163840=557056,720896=671744,1392640=98304} 2024-09-17T17:29:58.724 INFO:tasks.rados.rados.0.smithi102.stdout:129: writing smithi10238019-10 from 163840 to 720896 tid 1 2024-09-17T17:29:58.728 INFO:tasks.rados.rados.0.smithi102.stdout:129: writing smithi10238019-10 from 720896 to 1392640 tid 2 2024-09-17T17:29:58.729 INFO:tasks.rados.rados.0.smithi102.stdout:129: writing smithi10238019-10 from 1392640 to 1490944 tid 3 2024-09-17T17:29:58.729 INFO:tasks.rados.rados.0.smithi102.stdout:130: setattr oid 7 current snap is 8 2024-09-17T17:29:58.730 INFO:tasks.rados.rados.0.smithi102.stdout:127: finishing write tid 1 to smithi10238019-35 2024-09-17T17:29:58.730 INFO:tasks.rados.rados.0.smithi102.stdout:128: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-17T17:29:58.730 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 48825 2024-09-17T17:29:58.730 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 42318 2024-09-17T17:29:58.730 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:29:58.731 INFO:tasks.rados.rados.0.smithi102.stdout:127: finishing write tid 2 to smithi10238019-35 2024-09-17T17:29:58.731 INFO:tasks.rados.rados.0.smithi102.stdout:128: done (3 left) 2024-09-17T17:29:58.731 INFO:tasks.rados.rados.0.smithi102.stdout:131: snap_remove snap 3 2024-09-17T17:29:58.772 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.772 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:29:58.802 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:29:58.800+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:29:58.888 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:58.888 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:58.897 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:29:58.903 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:58.901+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:59.004 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:29:59.002+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:29:59.009 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.009 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.018 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:29:59.105 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:59.103+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:59.122 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.122 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.130 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:29:59.205 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:59.203+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:59.232 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.232 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.240 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:29:59.307 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:29:59.305+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:29:59.344 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.344 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.352 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:29:59.409 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:29:59.407+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:29:59.454 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.455 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.463 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:29:59.510 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:29:59.507+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:29:59.567 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.567 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.576 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:29:59.619 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:29:59.618+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:29:59.678 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.679 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.687 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:29:59.711 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:29:59.709+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:29:59.711 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:29:59.713 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:29:59.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:131: done (3 left) 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:132: copy_from oid 34 from oid 17 current snap is 8 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: finishing write tid 3 to smithi10238019-35 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: finishing write tid 4 to smithi10238019-35 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: finishing write tid 5 to smithi10238019-35 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: oid 35 updating version 0 to 26 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: oid 35 updating version 26 to 27 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: oid 35 updating version 27 to 28 2024-09-17T17:29:59.716 INFO:tasks.rados.rados.0.smithi102.stdout:127: oid 35 version 28 is already newer than 25 2024-09-17T17:29:59.717 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 35 v 28 (ObjNum 77 snap 8 seq_num 77) dirty exists 2024-09-17T17:29:59.717 INFO:tasks.rados.rados.0.smithi102.stdout:127: left oid 35 (ObjNum 77 snap 8 seq_num 77) 2024-09-17T17:29:59.718 INFO:tasks.rados.rados.0.smithi102.stdout:129: finishing write tid 1 to smithi10238019-10 2024-09-17T17:29:59.718 INFO:tasks.rados.rados.0.smithi102.stdout:129: finishing write tid 2 to smithi10238019-10 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: finishing write tid 3 to smithi10238019-10 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 18 (ObjNum 76 snap 4 seq_num 76) dirty exists 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: finishing write tid 4 to smithi10238019-10 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: finishing write tid 5 to smithi10238019-10 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: oid 10 updating version 0 to 36 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: oid 10 updating version 36 to 39 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: oid 10 version 39 is already newer than 38 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:129: oid 10 version 39 is already newer than 37 2024-09-17T17:29:59.719 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 10 v 39 (ObjNum 78 snap 8 seq_num 78) dirty exists 2024-09-17T17:29:59.720 INFO:tasks.rados.rados.0.smithi102.stdout:129: left oid 10 (ObjNum 78 snap 8 seq_num 78) 2024-09-17T17:29:59.720 INFO:tasks.rados.rados.0.smithi102.stdout:127: done (3 left) 2024-09-17T17:29:59.720 INFO:tasks.rados.rados.0.smithi102.stdout:129: done (2 left) 2024-09-17T17:29:59.720 INFO:tasks.rados.rados.0.smithi102.stdout:130: done (1 left) 2024-09-17T17:29:59.720 INFO:tasks.rados.rados.0.smithi102.stdout:133: rollback oid 2 current snap is 8 2024-09-17T17:29:59.720 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 2 to 6 2024-09-17T17:29:59.722 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 46 current snap is 8 2024-09-17T17:29:59.722 INFO:tasks.rados.rados.0.smithi102.stdout:134: seq_num 79 ranges {1966080=573440,2539520=425984,2965504=344064} 2024-09-17T17:29:59.731 INFO:tasks.rados.rados.0.smithi102.stdout:134: writing smithi10238019-46 from 1966080 to 2539520 tid 1 2024-09-17T17:29:59.734 INFO:tasks.rados.rados.0.smithi102.stdout:134: writing smithi10238019-46 from 2539520 to 2965504 tid 2 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:134: writing smithi10238019-46 from 2965504 to 3309568 tid 3 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:135: read oid 24 snap -1 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:135: expect (ObjNum 65 snap 3 seq_num 65) 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:136: rmattr oid 12 current snap is 8 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:136: done (4 left) 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:137: snap_create 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:133: finishing rollback tid 0 to smithi10238019-2 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:133: finishing rollback tid 1 to smithi10238019-2 2024-09-17T17:29:59.737 INFO:tasks.rados.rados.0.smithi102.stdout:133: finishing rollback tid 2 to smithi10238019-2 2024-09-17T17:29:59.738 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 2 v 26 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-17T17:29:59.742 INFO:tasks.rados.rados.0.smithi102.stdout:134: finishing write tid 1 to smithi10238019-46 2024-09-17T17:29:59.742 INFO:tasks.rados.rados.0.smithi102.stdout:134: finishing write tid 2 to smithi10238019-46 2024-09-17T17:29:59.749 INFO:tasks.rados.rados.0.smithi102.stdout:134: finishing write tid 3 to smithi10238019-46 2024-09-17T17:29:59.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:29:59.811 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:29:59.809+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:29:59.821 INFO:tasks.rados.rados.0.smithi102.stdout:134: finishing write tid 4 to smithi10238019-46 2024-09-17T17:29:59.821 INFO:tasks.rados.rados.0.smithi102.stdout:134: finishing write tid 5 to smithi10238019-46 2024-09-17T17:29:59.821 INFO:tasks.rados.rados.0.smithi102.stdout:134: oid 46 updating version 0 to 26 2024-09-17T17:29:59.821 INFO:tasks.rados.rados.0.smithi102.stdout:134: oid 46 version 26 is already newer than 25 2024-09-17T17:29:59.822 INFO:tasks.rados.rados.0.smithi102.stdout:134: oid 46 updating version 26 to 28 2024-09-17T17:29:59.822 INFO:tasks.rados.rados.0.smithi102.stdout:134: oid 46 version 28 is already newer than 27 2024-09-17T17:29:59.822 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 28 (ObjNum 79 snap 8 seq_num 79) dirty exists 2024-09-17T17:29:59.822 INFO:tasks.rados.rados.0.smithi102.stdout:134: left oid 46 (ObjNum 79 snap 8 seq_num 79) 2024-09-17T17:29:59.831 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.831 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.839 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:29:59.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:29:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:29:59.923 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:29:59.922+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:29:59.933 INFO:tasks.rados.rados.0.smithi102.stdout:132: finishing copy_from racing read to smithi10238019-34 2024-09-17T17:29:59.936 INFO:tasks.rados.rados.0.smithi102.stdout:132: finishing copy_from to smithi10238019-34 2024-09-17T17:29:59.936 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 34 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-17T17:29:59.942 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:29:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:29:59.964 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:29:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:29:59.984 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:29:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:29:59.985 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:29:59.985 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:29:59.993 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:29:59.997 INFO:tasks.rados.rados.0.smithi102.stdout:135: expect (ObjNum 65 snap 3 seq_num 65) 2024-09-17T17:29:59.998 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.005 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:00.005 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:00.013 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:00.011+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:00.014 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.030 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.102 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:00.102 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:00.111 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:00.114 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:00.112+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:00.215 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:00.213+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:00.254 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:00.254 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:00.263 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:00.317 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:00.315+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:00.328 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:132: done (4 left) 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:133: done (3 left) 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:134: done (2 left) 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:135: done (1 left) 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:137: done (0 left) 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:138: copy_from oid 27 from oid 19 current snap is 9 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 44 current snap is 9 2024-09-17T17:30:00.331 INFO:tasks.rados.rados.0.smithi102.stdout:139: seq_num 80 ranges {933888=589824,1523712=573440,2097152=196608} 2024-09-17T17:30:00.339 INFO:tasks.rados.rados.0.smithi102.stdout:139: writing smithi10238019-44 from 933888 to 1523712 tid 1 2024-09-17T17:30:00.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T17:30:00.343 INFO:tasks.rados.rados.0.smithi102.stdout:139: writing smithi10238019-44 from 1523712 to 2097152 tid 2 2024-09-17T17:30:00.344 INFO:tasks.rados.rados.0.smithi102.stdout:139: writing smithi10238019-44 from 2097152 to 2293760 tid 3 2024-09-17T17:30:00.344 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 15 current snap is 9 2024-09-17T17:30:00.344 INFO:tasks.rados.rados.0.smithi102.stdout:140: seq_num 81 ranges {245760=606208,851968=720896,1572864=49152} 2024-09-17T17:30:00.349 INFO:tasks.rados.rados.0.smithi102.stdout:140: writing smithi10238019-15 from 245760 to 851968 tid 1 2024-09-17T17:30:00.353 INFO:tasks.rados.rados.0.smithi102.stdout:140: writing smithi10238019-15 from 851968 to 1572864 tid 2 2024-09-17T17:30:00.354 INFO:tasks.rados.rados.0.smithi102.stdout:140: writing smithi10238019-15 from 1572864 to 1622016 tid 3 2024-09-17T17:30:00.354 INFO:tasks.rados.rados.0.smithi102.stdout:141: rollback oid 10 current snap is 9 2024-09-17T17:30:00.354 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 10 to 6 2024-09-17T17:30:00.354 INFO:tasks.rados.rados.0.smithi102.stdout:138: finishing copy_from racing read to smithi10238019-27 2024-09-17T17:30:00.354 INFO:tasks.rados.rados.0.smithi102.stdout:138: finishing copy_from to smithi10238019-27 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 27 v 5 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:139: finishing write tid 1 to smithi10238019-44 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:138: done (3 left) 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:142: read oid 18 snap -1 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:142: expect (ObjNum 17 snap 0 seq_num 17) 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:143: read oid 2 snap -1 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:143: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:144: rmattr oid 19 current snap is 9 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:144: done (5 left) 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:145: delete oid 16 current snap is 9 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:139: finishing write tid 2 to smithi10238019-44 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:139: finishing write tid 3 to smithi10238019-44 2024-09-17T17:30:00.355 INFO:tasks.rados.rados.0.smithi102.stdout:139: finishing write tid 4 to smithi10238019-44 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:140: finishing write tid 1 to smithi10238019-15 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:139: finishing write tid 5 to smithi10238019-44 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:139: oid 44 updating version 0 to 14 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:139: oid 44 updating version 14 to 16 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:139: oid 44 version 16 is already newer than 15 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:139: oid 44 version 16 is already newer than 13 2024-09-17T17:30:00.356 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 44 v 16 (ObjNum 80 snap 9 seq_num 80) dirty exists 2024-09-17T17:30:00.357 INFO:tasks.rados.rados.0.smithi102.stdout:139: left oid 44 (ObjNum 80 snap 9 seq_num 80) 2024-09-17T17:30:00.360 INFO:tasks.rados.rados.0.smithi102.stdout:141: finishing rollback tid 0 to smithi10238019-10 2024-09-17T17:30:00.361 INFO:tasks.rados.rados.0.smithi102.stdout:140: finishing write tid 2 to smithi10238019-15 2024-09-17T17:30:00.361 INFO:tasks.rados.rados.0.smithi102.stdout:142: expect (ObjNum 17 snap 0 seq_num 17) 2024-09-17T17:30:00.361 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.366 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.371 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.376 INFO:tasks.rados.rados.0.smithi102.stdout:140: finishing write tid 3 to smithi10238019-15 2024-09-17T17:30:00.376 INFO:tasks.rados.rados.0.smithi102.stdout:141: finishing rollback tid 1 to smithi10238019-10 2024-09-17T17:30:00.376 INFO:tasks.rados.rados.0.smithi102.stdout:141: finishing rollback tid 2 to smithi10238019-10 2024-09-17T17:30:00.376 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 10 v 47 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-17T17:30:00.377 INFO:tasks.rados.rados.0.smithi102.stdout:143: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-17T17:30:00.377 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.377 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:00.377 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1766 2024-09-17T17:30:00.383 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:00.383 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:00.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:00.417 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:00.415+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:00.518 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:00.516+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:00.539 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:00.539 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:00.539 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:00.539 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:00.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:00.618 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:00.617+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:00.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:00.749 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:00.747+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:00.814 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.127709554000001, 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020160569999999999, 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0499999999999999e-07 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683889+0000", 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000276748 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683926+0000", 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7307e-05 2024-09-17T17:30:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683937+0000", 2024-09-17T17:30:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0445000000000001e-05 2024-09-17T17:30:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684015+0000", 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8745999999999994e-05 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684028+0000", 2024-09-17T17:30:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2733e-05 2024-09-17T17:30:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015570569999999999 2024-09-17T17:30:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:30:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2616e-05 2024-09-17T17:30:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpRead(3.0s3 39/35 ECSubRead(tid=26, to_read={3:07ceeeba:::smithi10238019-36:head=0,147456,0}, subchunks={3:07ceeeba:::smithi10238019-36:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.125736142999999, 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001493276, 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2700000000000001e-07 2024-09-17T17:30:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685594+0000", 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6400000000000003e-06 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685681+0000", 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472999999999994e-05 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685691+0000", 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020000000000001e-05 2024-09-17T17:30:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685761+0000", 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9029000000000006e-05 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687078+0000", 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001317887 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 39/35 ECSubWrite(tid=48, reqid=client.4566.0:188, at_version=39'20, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685900+0000", 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.125421015000001, 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001607336, 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685900+0000", 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685900+0000", 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685901+0000", 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686027+0000", 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126308 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686073+0000", 2024-09-17T17:30:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5893999999999998e-05 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686083+0000", 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7610000000000002e-06 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686153+0000", 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9740999999999995e-05 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.686164+0000", 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1146e-05 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001303284 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687508+0000", 2024-09-17T17:30:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0250999999999998e-05 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.119437231999999, 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018516470000000001, 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691889+0000", 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6330000000000004e-06 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692145+0000", 2024-09-17T17:30:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025631599999999998 2024-09-17T17:30:00.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692190+0000", 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4969000000000003e-05 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692199+0000", 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4599999999999992e-06 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692277+0000", 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7676999999999995e-05 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692289+0000", 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693695+0000", 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014064679999999999 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693736+0000", 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-05 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 39/35 ECSubWrite(tid=51, reqid=client.4566.0:196, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.693817+0000", 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.117503985000001, 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018183520000000001, 2024-09-17T17:30:00.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693817+0000", 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693817+0000", 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693818+0000", 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3799999999999996e-07 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694010+0000", 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000192058 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694049+0000", 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.888e-05 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694057+0000", 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3520000000000007e-06 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694125+0000", 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7973999999999997e-05 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694139+0000", 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3295999999999999e-05 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695600+0000", 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001461279 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695636+0000", 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5775000000000002e-05 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 39/32 ECSubWrite(tid=15, reqid=client.4566.0:199, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:00.846 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.111105394999999, 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001846723, 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700460+0000", 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024369999999999999 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700559+0000", 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8998000000000003e-05 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700569+0000", 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9510000000000001e-06 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700641+0000", 2024-09-17T17:30:00.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2325999999999998e-05 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700656+0000", 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702016+0000", 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013601799999999999 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702063+0000", 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6384999999999997e-05 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=80, reqid=client.4566.0:274, at_version=42'13, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1034029319999998, 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003864363, 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.851 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707919+0000", 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708479+0000", 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055997299999999996 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708513+0000", 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4359000000000001e-05 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708524+0000", 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0346999999999999e-05 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708600+0000", 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999997e-05 2024-09-17T17:30:00.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708619+0000", 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8490000000000001e-05 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711752+0000", 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031327600000000001 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711783+0000", 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1035999999999999e-05 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=81, reqid=client.4566.0:275, at_version=42'14, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1010351270000003, 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025720560000000001, 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710287+0000", 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3300000000000001e-07 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710492+0000", 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020486500000000001 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710535+0000", 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3510999999999997e-05 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710545+0000", 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9869999999999994e-06 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710609+0000", 2024-09-17T17:30:00.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3379000000000004e-05 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710619+0000", 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0098000000000001e-05 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712823+0000", 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022037350000000001 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712858+0000", 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5648000000000002e-05 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=82, reqid=client.4566.0:276, at_version=42'15, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.711421+0000", 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0999000399999996, 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001452769, 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711421+0000", 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711421+0000", 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711422+0000", 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711443+0000", 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0661000000000001e-05 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711478+0000", 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4921e-05 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711497+0000", 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9786999999999999e-05 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711555+0000", 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7343999999999998e-05 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711565+0000", 2024-09-17T17:30:00.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0169999999999999e-05 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712862+0000", 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001296836 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712874+0000", 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2204999999999999e-05 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 42/32 ECSubWrite(tid=32, reqid=client.4566.0:278, at_version=42'23, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0881995910000004, 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001824928, 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:30:00.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723122+0000", 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2800000000000002e-07 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723213+0000", 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0434999999999997e-05 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723240+0000", 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7090999999999999e-05 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723250+0000", 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0141e-05 2024-09-17T17:30:00.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723328+0000", 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8588e-05 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.723341+0000", 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2568000000000001e-05 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.724879+0000", 2024-09-17T17:30:00.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015376599999999999 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.724947+0000", 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8016999999999996e-05 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 42/35 ECSubWrite(tid=86, reqid=client.4566.0:282, at_version=42'30, trim_to=0'0, roll_forward_to=41'28))", 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:00.870 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0776243579999996, 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002415295, 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:00.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733698+0000", 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2759999999999999e-06 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734149+0000", 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045118399999999998 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734186+0000", 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7109e-05 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734195+0000", 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0539999999999995e-06 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734270+0000", 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5165000000000003e-05 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734282+0000", 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2763e-05 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736069+0000", 2024-09-17T17:30:00.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017863530000000001 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736112+0000", 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3390999999999998e-05 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 42/32 ECSubWrite(tid=16, reqid=client.4566.0:289, at_version=42'12, trim_to=0'0, roll_forward_to=41'10))", 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.762220+0000", 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0491009289999997, 2024-09-17T17:30:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001441988, 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762220+0000", 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762220+0000", 2024-09-17T17:30:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762221+0000", 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762424+0000", 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000203069 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762454+0000", 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.972e-05 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762463+0000", 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6290000000000005e-06 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762536+0000", 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2893999999999997e-05 2024-09-17T17:30:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.762547+0000", 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0628999999999999e-05 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.763626+0000", 2024-09-17T17:30:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010797999999999999 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.763662+0000", 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5833000000000001e-05 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1185897220000003, 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020892049999999998, 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692761+0000", 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9167999999999999e-05 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692813+0000", 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2231999999999997e-05 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692825+0000", 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2075999999999999e-05 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692965+0000", 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013971200000000001 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692978+0000", 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2738999999999999e-05 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694786+0000", 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001808732 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694821+0000", 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4363000000000003e-05 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=37, reqid=osd.8.0:37, at_version=43'27, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1183280499999997, 2024-09-17T17:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001896823, 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693010+0000", 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6254999999999999e-05 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693046+0000", 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-05 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693055+0000", 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3480000000000004e-06 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693202+0000", 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000147369 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693211+0000", 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2499999999999995e-06 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694870+0000", 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016585759999999999 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694890+0000", 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0203999999999998e-05 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1175047410000003, 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001933894, 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693835+0000", 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7604999999999998e-05 2024-09-17T17:30:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693865+0000", 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0372999999999999e-05 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693877+0000", 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2143999999999999e-05 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693956+0000", 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9281000000000002e-05 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693968+0000", 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1100999999999999e-05 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.695713+0000", 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001745833 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.695750+0000", 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7015999999999999e-05 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.113754632, 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010715961, 2024-09-17T17:30:00.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697586+0000", 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8556999999999999e-05 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697629+0000", 2024-09-17T17:30:00.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2982999999999998e-05 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697637+0000", 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697698+0000", 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0534e-05 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697710+0000", 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1745e-05 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708248+0000", 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010538476999999999 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708283+0000", 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4431999999999998e-05 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1104705069999996, 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074300950000000003, 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700852+0000", 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1969999999999999e-06 2024-09-17T17:30:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700867+0000", 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4960000000000001e-05 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700917+0000", 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0285999999999999e-05 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700931+0000", 2024-09-17T17:30:00.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3332999999999999e-05 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701036+0000", 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105467 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701054+0000", 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7827999999999999e-05 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708242+0000", 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071881929999999998 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708281+0000", 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8831000000000001e-05 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1098869660000004, 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068976910000000001, 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701435+0000", 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701447+0000", 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2424e-05 2024-09-17T17:30:00.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1851e-05 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701527+0000", 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7069999999999992e-06 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701592+0000", 2024-09-17T17:30:00.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4825000000000004e-05 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701600+0000", 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-06 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708305+0000", 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067049680000000004 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708332+0000", 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6899000000000001e-05 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 48/32 ECSubWrite(tid=41, reqid=osd.8.0:40, at_version=0'0, trim_to=0'0, roll_forward_to=48'31))", 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0910973180000001, 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77214092400000001, 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720235+0000", 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0353999999999999e-05 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491642+0000", 2024-09-17T17:30:00.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77140694700000001 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491651+0000", 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491754+0000", 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010309 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491768+0000", 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144999999999999e-05 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492327+0000", 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000558998 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492365+0000", 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969000000000002e-05 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 49/32 ECSubWrite(tid=27, reqid=client.4566.0:372, at_version=49'20, trim_to=0'0, roll_forward_to=48'18))", 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:00.349823+0000", 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.46149864899999998, 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017060090000000001, 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.349823+0000", 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.349823+0000", 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.349823+0000", 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:30:00.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.350143+0000", 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031995599999999998 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.350171+0000", 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8149000000000001e-05 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.350177+0000", 2024-09-17T17:30:00.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0680000000000002e-06 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.350249+0000", 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1795e-05 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.350261+0000", 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1929999999999999e-05 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.351499+0000", 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001238105 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.351529+0000", 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9753000000000001e-05 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:00.918 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:00.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:00.921 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:00.922 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:140: finishing write tid 4 to smithi10238019-15 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:139: done (5 left) 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:141: done (4 left) 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:142: done (3 left) 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:143: done (2 left) 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:145: done (1 left) 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:146: copy_from oid 50 from oid 49 current snap is 9 2024-09-17T17:30:00.922 INFO:tasks.rados.rados.0.smithi102.stdout:140: finishing write tid 5 to smithi10238019-15 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:140: oid 15 updating version 0 to 21 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:140: oid 15 version 21 is already newer than 20 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:140: oid 15 updating version 21 to 22 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:140: oid 15 updating version 22 to 23 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 15 v 23 (ObjNum 81 snap 9 seq_num 81) dirty exists 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:140: left oid 15 (ObjNum 81 snap 9 seq_num 81) 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:140: done (1 left) 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:147: snap_create 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:146: finishing copy_from racing read to smithi10238019-50 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:146: finishing copy_from to smithi10238019-50 2024-09-17T17:30:00.923 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 50 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-17T17:30:00.924 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:00.924 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:00.925 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:00.925 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:00.918+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:00.925 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:00.926 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:00.926 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:00.926 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:01.022 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:01.019+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:01.064 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:01.064 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:01.068 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:01.069 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:01.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:01.086 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:01.106 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:01.122 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:01.120+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:01.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:01.127+0000 7f21f67fc640 0 -- 172.21.15.7:0/4231160001 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f21f8221660 msgr2=0x7f21f8223a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:01.185 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:01.185 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:01.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:01.223 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:01.221+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:01.324 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:01.322+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:01.329 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T17:30:01.332 INFO:tasks.rados.rados.0.smithi102.stdout:146: done (1 left) 2024-09-17T17:30:01.332 INFO:tasks.rados.rados.0.smithi102.stdout:147: done (0 left) 2024-09-17T17:30:01.332 INFO:tasks.rados.rados.0.smithi102.stdout:148: setattr oid 6 current snap is 10 2024-09-17T17:30:01.333 INFO:tasks.rados.rados.0.smithi102.stdout:149: rmattr oid 4 current snap is 10 2024-09-17T17:30:01.333 INFO:tasks.rados.rados.0.smithi102.stdout:149: done (1 left) 2024-09-17T17:30:01.333 INFO:tasks.rados.rados.0.smithi102.stdout:150: read oid 25 snap -1 2024-09-17T17:30:01.333 INFO:tasks.rados.rados.0.smithi102.stdout:150: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-17T17:30:01.333 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 18 current snap is 10 2024-09-17T17:30:01.333 INFO:tasks.rados.rados.0.smithi102.stdout:151: seq_num 83 ranges {294912=638976,933888=622592,1556480=147456} 2024-09-17T17:30:01.338 INFO:tasks.rados.rados.0.smithi102.stdout:151: writing smithi10238019-18 from 294912 to 933888 tid 1 2024-09-17T17:30:01.342 INFO:tasks.rados.rados.0.smithi102.stdout:151: writing smithi10238019-18 from 933888 to 1556480 tid 2 2024-09-17T17:30:01.344 INFO:tasks.rados.rados.0.smithi102.stdout:151: writing smithi10238019-18 from 1556480 to 1703936 tid 3 2024-09-17T17:30:01.344 INFO:tasks.rados.rados.0.smithi102.stdout:152: copy_from oid 29 from oid 13 current snap is 10 2024-09-17T17:30:01.344 INFO:tasks.rados.rados.0.smithi102.stdout:150: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-17T17:30:01.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T17:30:01.346 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 328220 2024-09-17T17:30:01.347 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:01.347 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:01.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:01.347 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:01.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=14, reqid=client.4566.0:184, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.665091946, 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001332981, 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678506+0000", 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6999999999999999e-07 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678530+0000", 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3696000000000001e-05 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678576+0000", 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5528999999999997e-05 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678588+0000", 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2527999999999999e-05 2024-09-17T17:30:01.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678664+0000", 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5785999999999996e-05 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.678678+0000", 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.436e-05 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.679792+0000", 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001113246 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.679839+0000", 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7666000000000001e-05 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:01.352 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.658722658, 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002820571, 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684890+0000", 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3499000000000001e-05 2024-09-17T17:30:01.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685111+0000", 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022111 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685121+0000", 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7070000000000004e-06 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685199+0000", 2024-09-17T17:30:01.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8145999999999993e-05 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685209+0000", 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0138e-05 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687657+0000", 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024481260000000001 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687696+0000", 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.909e-05 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/35 ECSubWrite(tid=11, reqid=osd.1.0:10, at_version=0'0, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.658149999000001, 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002246264, 2024-09-17T17:30:01.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685449+0000", 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0800000000000004e-07 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685461+0000", 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1664999999999999e-05 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685533+0000", 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2149000000000003e-05 2024-09-17T17:30:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685542+0000", 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685619+0000", 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7186000000000003e-05 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687653+0000", 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020252030000000002 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687694+0000", 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1072000000000002e-05 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 39/32 ECSubWrite(tid=13, reqid=osd.14.0:13, at_version=39'10, trim_to=0'0, roll_forward_to=37'8))", 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691662+0000", 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.651935706, 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011926249999999999, 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691662+0000", 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691662+0000", 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691663+0000", 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.84e-07 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691694+0000", 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609000000000002e-05 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691741+0000", 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6326999999999999e-05 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691751+0000", 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:01.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691831+0000", 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0129999999999993e-05 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691844+0000", 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2903e-05 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692817+0000", 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00097266099999999999 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692855+0000", 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8040999999999997e-05 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:01.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.651635197000001, 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022546670000000001, 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8700000000000002e-07 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692554+0000", 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000590721 2024-09-17T17:30:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692590+0000", 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6350000000000003e-05 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692602+0000", 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692708+0000", 2024-09-17T17:30:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106835 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692718+0000", 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694181+0000", 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014632149999999999 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694218+0000", 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6730999999999998e-05 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=51, reqid=client.4566.0:196, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.693896+0000", 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.649702572000001, 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016845129999999999, 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693896+0000", 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693896+0000", 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693897+0000", 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0079999999999999e-06 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694141+0000", 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000244856 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694178+0000", 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6096999999999997e-05 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694186+0000", 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8740000000000001e-06 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694255+0000", 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8415000000000001e-05 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694264+0000", 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9400000000000008e-06 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695539+0000", 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001274982 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695580+0000", 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1341000000000003e-05 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.375 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=54, reqid=client.4566.0:197, at_version=39'8, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.694726+0000", 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.648871854999999, 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010910270000000001, 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694726+0000", 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694726+0000", 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694727+0000", 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2999999999999999e-07 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694746+0000", 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9228e-05 2024-09-17T17:30:01.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694783+0000", 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6044000000000003e-05 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694792+0000", 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3370000000000006e-06 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694858+0000", 2024-09-17T17:30:01.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5899e-05 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694868+0000", 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0372e-05 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695783+0000", 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091487700000000003 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.695817+0000", 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.434e-05 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 39/32 ECSubWrite(tid=14, reqid=osd.14.0:13, at_version=0'0, trim_to=0'0, roll_forward_to=39'10))", 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698072+0000", 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.645526290999999, 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013017510000000001, 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698072+0000", 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698072+0000", 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698073+0000", 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7700000000000004e-07 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698082+0000", 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1160000000000005e-06 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698109+0000", 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6973000000000001e-05 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698117+0000", 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3259999999999996e-06 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698169+0000", 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2083000000000003e-05 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698177+0000", 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4340000000000003e-06 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699332+0000", 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011553259999999999 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699374+0000", 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1715999999999998e-05 2024-09-17T17:30:01.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/35 ECSubWrite(tid=12, reqid=client.4566.0:203, at_version=39'9, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.705385+0000", 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.638213021, 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017198960000000001, 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705385+0000", 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705385+0000", 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705386+0000", 2024-09-17T17:30:01.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0809999999999999e-06 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705923+0000", 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000536949 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705968+0000", 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4969999999999998e-05 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705979+0000", 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0502e-05 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706074+0000", 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4996999999999999e-05 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706090+0000", 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5948999999999999e-05 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707076+0000", 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098629800000000004 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707105+0000", 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.915e-05 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 39/35 ECSubWrite(tid=27, reqid=client.4566.0:209, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.632519087, 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011625979999999999, 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711079+0000", 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711105+0000", 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5629000000000001e-05 2024-09-17T17:30:01.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711164+0000", 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8548000000000003e-05 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711174+0000", 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0292000000000001e-05 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711311+0000", 2024-09-17T17:30:01.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137324 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711321+0000", 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0132000000000001e-05 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712206+0000", 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00088499800000000001 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712242+0000", 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5395000000000002e-05 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 39/32 ECSubWrite(tid=34, reqid=client.4566.0:207, at_version=39'12, trim_to=0'0, roll_forward_to=36'10))", 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.714563+0000", 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.629035305, 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014888449999999999, 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714563+0000", 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714563+0000", 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714564+0000", 2024-09-17T17:30:01.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714823+0000", 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025957399999999999 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714857+0000", 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4078999999999997e-05 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714863+0000", 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6760000000000004e-06 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714929+0000", 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6049000000000004e-05 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714940+0000", 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1358999999999999e-05 2024-09-17T17:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716004+0000", 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010639289999999999 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716052+0000", 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7555000000000001e-05 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.623060639, 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016256700000000001, 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720538+0000", 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.720975+0000", 2024-09-17T17:30:01.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043686899999999999 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721002+0000", 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7030999999999999e-05 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721008+0000", 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.942e-06 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721060+0000", 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1916000000000002e-05 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.721066+0000", 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722130+0000", 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010640070000000001 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.722163+0000", 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3198999999999998e-05 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 43/35 ECSubWrite(tid=43, reqid=osd.12.0:43, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.696596+0000", 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.6470025589999997, 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001458403, 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.403 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696596+0000", 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696596+0000", 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696596+0000", 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696620+0000", 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3866000000000002e-05 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696679+0000", 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9159000000000001e-05 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696689+0000", 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6339999999999996e-06 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696755+0000", 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6500000000000004e-05 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696765+0000", 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0200000000000001e-05 2024-09-17T17:30:01.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698016+0000", 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001250391 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698054+0000", 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8266000000000003e-05 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697950+0000", 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.6456482210000001, 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001462522, 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697950+0000", 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697950+0000", 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697950+0000", 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.35e-07 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697970+0000", 2024-09-17T17:30:01.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0208999999999999e-05 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698009+0000", 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8356999999999999e-05 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698016+0000", 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6879999999999999e-06 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698083+0000", 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6904999999999997e-05 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698093+0000", 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6099999999999995e-06 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699377+0000", 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012839500000000001 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699412+0000", 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5568000000000002e-05 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 43/32 ECSubWrite(tid=55, reqid=osd.11.0:55, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.701608+0000", 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.641989948, 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012118299999999999, 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701608+0000", 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701608+0000", 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701609+0000", 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8699999999999995e-07 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701630+0000", 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1501999999999998e-05 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.702073+0000", 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044237399999999999 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.702080+0000", 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.702138+0000", 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7417000000000003e-05 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.702147+0000", 2024-09-17T17:30:01.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2599999999999994e-06 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.702787+0000", 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000640141 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.702820+0000", 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2968e-05 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 43/32 ECSubWrite(tid=15, reqid=client.4566.0:307, at_version=43'12, trim_to=0'0, roll_forward_to=39'10))", 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.971106+0000", 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.3724924019999998, 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001097432, 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971106+0000", 2024-09-17T17:30:01.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971106+0000", 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971106+0000", 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.08e-07 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971123+0000", 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7113999999999999e-05 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971159+0000", 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5506000000000002e-05 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971175+0000", 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6912999999999999e-05 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971251+0000", 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5464999999999997e-05 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971261+0000", 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0244999999999999e-05 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972165+0000", 2024-09-17T17:30:01.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090414499999999997 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972203+0000", 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7935999999999997e-05 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 43/35 ECSubWrite(tid=59, reqid=client.4566.0:313, at_version=43'11, trim_to=0'0, roll_forward_to=42'8))", 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.982380+0000", 2024-09-17T17:30:01.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.3612183560000002, 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012083599999999999, 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982380+0000", 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982380+0000", 2024-09-17T17:30:01.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982381+0000", 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1370000000000001e-06 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982713+0000", 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033242900000000003 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982742+0000", 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8958e-05 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982751+0000", 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5310000000000001e-06 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982812+0000", 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1069999999999996e-05 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.982823+0000", 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-05 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.983562+0000", 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073892000000000001 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.983588+0000", 2024-09-17T17:30:01.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6375000000000001e-05 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 49/35 ECSubWrite(tid=39, reqid=client.4566.0:365, at_version=49'28, trim_to=0'0, roll_forward_to=48'26))", 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:00.338656+0000", 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0049422720000001, 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017765330000000001, 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.426 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338656+0000", 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338656+0000", 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338656+0000", 2024-09-17T17:30:01.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999999999999999e-07 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338870+0000", 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000213709 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338890+0000", 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9850000000000001e-05 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338897+0000", 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338972+0000", 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4373000000000001e-05 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.338979+0000", 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7959999999999996e-06 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.340394+0000", 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001414909 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.340432+0000", 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8095e-05 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.430 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 49/35 ECSubWrite(tid=20, reqid=client.4566.0:367, at_version=49'13, trim_to=0'0, roll_forward_to=39'11))", 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:00.342862+0000", 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.00073594, 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001312145, 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.342862+0000", 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.342862+0000", 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.342863+0000", 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.343097+0000", 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023464800000000001 2024-09-17T17:30:01.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.343123+0000", 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5466e-05 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.343131+0000", 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4910000000000005e-06 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.343180+0000", 2024-09-17T17:30:01.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9106e-05 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.343187+0000", 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4420000000000003e-06 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.344135+0000", 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000948134 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.344174+0000", 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9456000000000003e-05 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 49/35 ECSubWrite(tid=84, reqid=client.4566.0:386, at_version=49'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:00.358154+0000", 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.985443807, 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001214878, 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358154+0000", 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358154+0000", 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358155+0000", 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6999999999999999e-07 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358184+0000", 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9366e-05 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358310+0000", 2024-09-17T17:30:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126477 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358317+0000", 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.041e-06 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358397+0000", 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9721000000000005e-05 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.358403+0000", 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8370000000000002e-06 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.359346+0000", 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094313700000000003 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.359369+0000", 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3129000000000002e-05 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.440 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:01.441 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:01.442 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 182848 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 342602 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 6 v 32 (ObjNum 75 snap 4 seq_num 75) dirty exists 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:151: finishing write tid 1 to smithi10238019-18 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:151: finishing write tid 2 to smithi10238019-18 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:148: done (3 left) 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:150: done (2 left) 2024-09-17T17:30:01.443 INFO:tasks.rados.rados.0.smithi102.stdout:153: rollback oid 7 current snap is 10 2024-09-17T17:30:01.444 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 7 to 7 2024-09-17T17:30:01.444 INFO:tasks.rados.rados.0.smithi102.stdout:154: snap_remove snap 5 2024-09-17T17:30:01.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:01.444 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:01.446 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:01.439+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:01.542 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:01.539+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:01.590 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:01.591 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:01.591 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:01.591 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:01.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:01.643 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:01.641+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:01.706 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:01.706 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:01.706 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:01.707 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:01.707 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:01.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:01.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:30:01.743 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:01.740+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.137149506, 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:30:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=19, reqid=osd.15.0:19, at_version=39'15, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.133474972, 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001026447, 2024-09-17T17:30:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688331+0000", 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7700000000000001e-07 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688373+0000", 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2141999999999997e-05 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688428+0000", 2024-09-17T17:30:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4759999999999997e-05 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688437+0000", 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5699999999999993e-06 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688514+0000", 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6958000000000004e-05 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.688524+0000", 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.689315+0000", 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00079100500000000001 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.689358+0000", 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2840000000000003e-05 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=20, reqid=client.4566.0:191, at_version=39'17, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.131615172, 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001787521, 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690192+0000", 2024-09-17T17:30:01.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T17:30:01.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690618+0000", 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042645600000000002 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690656+0000", 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8214999999999999e-05 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690666+0000", 2024-09-17T17:30:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5419999999999994e-06 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690744+0000", 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8418999999999996e-05 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690754+0000", 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9329999999999996e-06 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691939+0000", 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001184198 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691978+0000", 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9845999999999998e-05 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.123613337, 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:30:01.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.121387478000001, 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:01.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.120963399000001, 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:01.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:30:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:30:01.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.120869097, 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:30:01.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:30:01.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.120244134, 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015435340000000001, 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:01.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701563+0000", 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.155e-06 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701569+0000", 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701604+0000", 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5148999999999998e-05 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701612+0000", 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9559999999999997e-06 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703084+0000", 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014716270000000001 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703105+0000", 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1766000000000001e-05 2024-09-17T17:30:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.120006846000001, 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001340387, 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701800+0000", 2024-09-17T17:30:01.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2900000000000002e-07 2024-09-17T17:30:01.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701806+0000", 2024-09-17T17:30:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5099999999999998e-06 2024-09-17T17:30:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701957+0000", 2024-09-17T17:30:01.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000151175 2024-09-17T17:30:01.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701965+0000", 2024-09-17T17:30:01.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-17T17:30:01.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703127+0000", 2024-09-17T17:30:01.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011621019999999999 2024-09-17T17:30:01.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703140+0000", 2024-09-17T17:30:01.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.295e-05 2024-09-17T17:30:01.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:01.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.702013+0000", 2024-09-17T17:30:01.878 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.119792890999999, 2024-09-17T17:30:01.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001197989, 2024-09-17T17:30:01.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.879 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702013+0000", 2024-09-17T17:30:01.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702013+0000", 2024-09-17T17:30:01.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702014+0000", 2024-09-17T17:30:01.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T17:30:01.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702021+0000", 2024-09-17T17:30:01.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0269999999999998e-06 2024-09-17T17:30:01.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702067+0000", 2024-09-17T17:30:01.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6007999999999997e-05 2024-09-17T17:30:01.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702075+0000", 2024-09-17T17:30:01.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000007e-06 2024-09-17T17:30:01.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703188+0000", 2024-09-17T17:30:01.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001112315 2024-09-17T17:30:01.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703211+0000", 2024-09-17T17:30:01.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3448e-05 2024-09-17T17:30:01.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.702100+0000", 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.119705680999999, 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001135863, 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702100+0000", 2024-09-17T17:30:01.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702100+0000", 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702101+0000", 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0800000000000003e-07 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702106+0000", 2024-09-17T17:30:01.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0529999999999997e-06 2024-09-17T17:30:01.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702144+0000", 2024-09-17T17:30:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8058000000000001e-05 2024-09-17T17:30:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702157+0000", 2024-09-17T17:30:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.289e-05 2024-09-17T17:30:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703226+0000", 2024-09-17T17:30:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010690949999999999 2024-09-17T17:30:01.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703236+0000", 2024-09-17T17:30:01.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.859e-06 2024-09-17T17:30:01.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.7s1 39/32 ECSubWrite(tid=57, reqid=client.4566.0:182, at_version=39'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-17T17:30:01.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:01.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.118373872999999, 2024-09-17T17:30:01.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00185337, 2024-09-17T17:30:01.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.896 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:01.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:01.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703433+0000", 2024-09-17T17:30:01.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4300000000000002e-07 2024-09-17T17:30:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704005+0000", 2024-09-17T17:30:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057213000000000001 2024-09-17T17:30:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704034+0000", 2024-09-17T17:30:01.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8662000000000001e-05 2024-09-17T17:30:01.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704039+0000", 2024-09-17T17:30:01.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1170000000000003e-06 2024-09-17T17:30:01.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704101+0000", 2024-09-17T17:30:01.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1818999999999999e-05 2024-09-17T17:30:01.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704111+0000", 2024-09-17T17:30:01.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0115000000000001e-05 2024-09-17T17:30:01.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705248+0000", 2024-09-17T17:30:01.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011371420000000001 2024-09-17T17:30:01.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705286+0000", 2024-09-17T17:30:01.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7542000000000001e-05 2024-09-17T17:30:01.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=35, reqid=osd.3.0:35, at_version=43'18, trim_to=0'0, roll_forward_to=42'16))", 2024-09-17T17:30:01.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:01.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1244183369999998, 2024-09-17T17:30:01.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001758098, 2024-09-17T17:30:01.906 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:01.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:01.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:01.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:30:01.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697411+0000", 2024-09-17T17:30:01.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2937000000000001e-05 2024-09-17T17:30:01.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697449+0000", 2024-09-17T17:30:01.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519000000000002e-05 2024-09-17T17:30:01.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697458+0000", 2024-09-17T17:30:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-17T17:30:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697532+0000", 2024-09-17T17:30:01.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4147999999999995e-05 2024-09-17T17:30:01.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697542+0000", 2024-09-17T17:30:01.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.893e-06 2024-09-17T17:30:01.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699107+0000", 2024-09-17T17:30:01.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015656050000000001 2024-09-17T17:30:01.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699146+0000", 2024-09-17T17:30:01.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8525000000000001e-05 2024-09-17T17:30:01.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.bs4 43/32 ECSubWrite(tid=92, reqid=osd.13.0:92, at_version=43'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-17T17:30:01.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:01.917 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1239294060000002, 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019485469999999999, 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697917+0000", 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9153999999999998e-05 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697948+0000", 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1538000000000002e-05 2024-09-17T17:30:01.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697957+0000", 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698771+0000", 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081437600000000001 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698781+0000", 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9590000000000007e-06 2024-09-17T17:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699776+0000", 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099482900000000003 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699825+0000", 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9280000000000003e-05 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=37, reqid=client.4566.0:303, at_version=43'20, trim_to=0'0, roll_forward_to=43'18))", 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.855112836, 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013508630000000001, 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966695+0000", 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8390000000000001e-06 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966972+0000", 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027734400000000001 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967017+0000", 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4558000000000003e-05 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967027+0000", 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0431999999999999e-05 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967083+0000", 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5686999999999997e-05 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967090+0000", 2024-09-17T17:30:01.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.861e-06 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.968020+0000", 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093037199999999995 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.968044+0000", 2024-09-17T17:30:01.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3770000000000001e-05 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=38, reqid=client.4566.0:304, at_version=43'21, trim_to=0'0, roll_forward_to=43'18))", 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.972172+0000", 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.8496339329999998, 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00121889, 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.930 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972172+0000", 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972172+0000", 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972173+0000", 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972446+0000", 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027284500000000002 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972483+0000", 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6782000000000001e-05 2024-09-17T17:30:01.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972493+0000", 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265000000000001e-05 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972558+0000", 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4723999999999999e-05 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972569+0000", 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1003e-05 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.973353+0000", 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078411099999999999 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.973391+0000", 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8177999999999999e-05 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=54, reqid=client.4566.0:390, at_version=50'32, trim_to=0'0, roll_forward_to=48'30))", 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.48558380400000001, 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001333426, 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336223+0000", 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T17:30:01.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336344+0000", 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121086 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336381+0000", 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6823000000000003e-05 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336391+0000", 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0725e-05 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336445+0000", 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3505999999999999e-05 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336452+0000", 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4749999999999996e-06 2024-09-17T17:30:01.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.337528+0000", 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010753760000000001 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.337556+0000", 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7937999999999999e-05 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=55, reqid=client.4566.0:394, at_version=50'34, trim_to=0'0, roll_forward_to=48'30))", 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.48119092699999999, 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014379869999999999, 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340617+0000", 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-17T17:30:01.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340887+0000", 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027011199999999998 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340937+0000", 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0489999999999999e-05 2024-09-17T17:30:01.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340949+0000", 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2054e-05 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.341016+0000", 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7136000000000002e-05 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.341026+0000", 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.342022+0000", 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099594500000000003 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.342053+0000", 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1619999999999999e-05 2024-09-17T17:30:01.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=56, reqid=client.4566.0:395, at_version=50'35, trim_to=0'0, roll_forward_to=50'32))", 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.47769446700000001, 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013727380000000001, 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344113+0000", 2024-09-17T17:30:01.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344333+0000", 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022061300000000001 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344418+0000", 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4672000000000005e-05 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344426+0000", 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.79e-06 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344496+0000", 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0178000000000005e-05 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344506+0000", 2024-09-17T17:30:01.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0299000000000001e-05 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.345448+0000", 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094149200000000004 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.345484+0000", 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6652000000000001e-05 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=57, reqid=client.4566.0:396, at_version=50'36, trim_to=0'0, roll_forward_to=50'34))", 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.346307+0000", 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.47549901300000003, 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010155489999999999, 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:01.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346307+0000", 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346307+0000", 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346308+0000", 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346350+0000", 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1508999999999998e-05 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:01.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346383+0000", 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3738000000000001e-05 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346394+0000", 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0283e-05 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:01.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346458+0000", 2024-09-17T17:30:01.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4512999999999997e-05 2024-09-17T17:30:01.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:01.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.346470+0000", 2024-09-17T17:30:01.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1806e-05 2024-09-17T17:30:01.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:01.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.347287+0000", 2024-09-17T17:30:01.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081725700000000005 2024-09-17T17:30:01.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:01.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:01.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:01.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.347323+0000", 2024-09-17T17:30:01.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5497999999999998e-05 2024-09-17T17:30:01.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:01.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:01.957 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:01.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:01.961 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:01.957+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:02.060 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:02.058+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:02.068 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:02.069 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:02.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:02.161 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:02.159+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:02.186 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:02.187 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:02.187 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:02.187 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:02.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:02.262 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:02.260+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:02.303 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.618184523, 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:30:02.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:30:02.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:30:02.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:30:02.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s1 39/35 ECSubWrite(tid=11, reqid=osd.1.0:10, at_version=0'0, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685357+0000", 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.61585217, 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018332299999999999, 2024-09-17T17:30:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685357+0000", 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685357+0000", 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685358+0000", 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.032e-06 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685371+0000", 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2689e-05 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685408+0000", 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6888000000000003e-05 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685418+0000", 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8560000000000002e-06 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685672+0000", 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025456600000000002 2024-09-17T17:30:02.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685683+0000", 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0806e-05 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687154+0000", 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014711539999999999 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687191+0000", 2024-09-17T17:30:02.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6239000000000003e-05 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s4 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.609059157000001, 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022843400000000002, 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692150+0000", 2024-09-17T17:30:02.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-07 2024-09-17T17:30:02.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692605+0000", 2024-09-17T17:30:02.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045479800000000002 2024-09-17T17:30:02.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692645+0000", 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0033000000000001e-05 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692655+0000", 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0156999999999999e-05 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692715+0000", 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9502000000000001e-05 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692721+0000", 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.105e-06 2024-09-17T17:30:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694369+0000", 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016482650000000001 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694435+0000", 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5369000000000004e-05 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:209 3.9s0 3:96cbaefd:::smithi10238019-46:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710441+0000", 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.590768795000001, 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064948369999999998, 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 209 2024-09-17T17:30:02.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710441+0000", 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710441+0000", 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710442+0000", 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4619999999999999e-06 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710449+0000", 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710461+0000", 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2629000000000001e-05 2024-09-17T17:30:02.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710467+0000", 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8560000000000003e-06 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710519+0000", 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1801000000000001e-05 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710596+0000", 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7019000000000002e-05 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711059+0000", 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046334700000000002 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714258+0000", 2024-09-17T17:30:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031989259999999999 2024-09-17T17:30:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716853+0000", 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002594369 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.716936+0000", 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2948000000000006e-05 2024-09-17T17:30:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:210 3.9s0 3:96cbaefd:::smithi10238019-46:head [rollback 3] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.590724985, 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065835980000000004, 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 210 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710485+0000", 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710487+0000", 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8369999999999999e-06 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710492+0000", 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5860000000000004e-06 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710496+0000", 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3239999999999999e-06 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711515+0000", 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010192700000000001 2024-09-17T17:30:02.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711606+0000", 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1471000000000006e-05 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712163+0000", 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055688199999999997 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714262+0000", 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020989889999999999 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717028+0000", 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027660889999999998 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717068+0000", 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9740999999999998e-05 2024-09-17T17:30:02.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:211 3.9s0 3:96cbaefd:::smithi10238019-46:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.590695653999999, 2024-09-17T17:30:02.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068972499999999997, 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 211 2024-09-17T17:30:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0400000000000002e-07 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710516+0000", 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5269999999999999e-06 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710520+0000", 2024-09-17T17:30:02.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4059999999999998e-06 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710524+0000", 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.518e-06 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712807+0000", 2024-09-17T17:30:02.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002283322 2024-09-17T17:30:02.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713024+0000", 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216629 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713484+0000", 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046035200000000003 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714391+0000", 2024-09-17T17:30:02.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090667600000000003 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717343+0000", 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002952863 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717411+0000", 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7652999999999998e-05 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:212 3.0s0 3:0b65e655:::smithi10238019-14:head [delete] snapc 3=[3] ondisk+write+known_if_redirected+full_force+supports_pool_eio e39)", 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.590680608, 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002126724, 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 212 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710529+0000", 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710530+0000", 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1349999999999999e-06 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710538+0000", 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6930000000000007e-06 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710544+0000", 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0920000000000003e-06 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710600+0000", 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5315999999999997e-05 2024-09-17T17:30:02.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710660+0000", 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0439000000000002e-05 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711125+0000", 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046455200000000002 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712320+0000", 2024-09-17T17:30:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001195577 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712597+0000", 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027682199999999997 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712656+0000", 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8684000000000003e-05 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 39/35 ECSubWriteReply(tid=27, last_complete=39'6, committed=1, applied=1))", 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.711934+0000", 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.589275899, 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017909989999999999, 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711934+0000", 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711934+0000", 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711935+0000", 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0500000000000002e-07 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711940+0000", 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1399999999999999e-06 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711973+0000", 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3061000000000001e-05 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.711981+0000", 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4290000000000001e-06 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713707+0000", 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001725643 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713725+0000", 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7821000000000001e-05 2024-09-17T17:30:02.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds3 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.6083193940000005, 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017461569999999999, 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692890+0000", 2024-09-17T17:30:02.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.98e-07 2024-09-17T17:30:02.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692920+0000", 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9940000000000001e-05 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692998+0000", 2024-09-17T17:30:02.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7701999999999996e-05 2024-09-17T17:30:02.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693009+0000", 2024-09-17T17:30:02.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1277999999999999e-05 2024-09-17T17:30:02.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693076+0000", 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6647000000000001e-05 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693087+0000", 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0712e-05 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694596+0000", 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015095600000000001 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694636+0000", 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0120000000000002e-05 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds3 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.693844+0000", 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.6073651670000002, 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025301949999999998, 2024-09-17T17:30:02.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693844+0000", 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693844+0000", 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693845+0000", 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693860+0000", 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4789e-05 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693885+0000", 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5157e-05 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0099999999999995e-06 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693969+0000", 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6512000000000002e-05 2024-09-17T17:30:02.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693981+0000", 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.145e-05 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696340+0000", 2024-09-17T17:30:02.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002359566 2024-09-17T17:30:02.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.696375+0000", 2024-09-17T17:30:02.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4076999999999999e-05 2024-09-17T17:30:02.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 43/35 ECSubWrite(tid=54, reqid=osd.11.0:54, at_version=43'10, trim_to=0'0, roll_forward_to=42'8))", 2024-09-17T17:30:02.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:30:02.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5972249720000002, 2024-09-17T17:30:02.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030555130000000002, 2024-09-17T17:30:02.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.374 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:30:02.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:30:02.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.703985+0000", 2024-09-17T17:30:02.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.66e-07 2024-09-17T17:30:02.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704004+0000", 2024-09-17T17:30:02.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9400999999999999e-05 2024-09-17T17:30:02.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706156+0000", 2024-09-17T17:30:02.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021521719999999999 2024-09-17T17:30:02.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706167+0000", 2024-09-17T17:30:02.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0804e-05 2024-09-17T17:30:02.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706242+0000", 2024-09-17T17:30:02.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5234000000000005e-05 2024-09-17T17:30:02.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706253+0000", 2024-09-17T17:30:02.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0755e-05 2024-09-17T17:30:02.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707001+0000", 2024-09-17T17:30:02.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074821000000000002 2024-09-17T17:30:02.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707040+0000", 2024-09-17T17:30:02.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8770999999999999e-05 2024-09-17T17:30:02.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 43/32 ECSubWrite(tid=55, reqid=osd.11.0:55, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-17T17:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5972042269999998, 2024-09-17T17:30:02.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039008089999999999, 2024-09-17T17:30:02.384 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.384 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:30:02.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704005+0000", 2024-09-17T17:30:02.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6999999999999999e-07 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.704014+0000", 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2509999999999994e-06 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706197+0000", 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021834990000000002 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706204+0000", 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3439999999999997e-06 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706345+0000", 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000141488 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.706355+0000", 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5489999999999995e-06 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707877+0000", 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001522175 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.707906+0000", 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9332999999999999e-05 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:30:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5872313, 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5549296569999997, 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:30:02.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.396 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:02.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:30:02.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:30:02.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:30:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5530051550000001, 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:30:02.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s1 43/32 ECSubWrite(tid=15, reqid=client.4566.0:307, at_version=43'12, trim_to=0'0, roll_forward_to=39'10))", 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.3300433390000004, 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001720982, 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971166+0000", 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971190+0000", 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-05 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971224+0000", 2024-09-17T17:30:02.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3748999999999998e-05 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971231+0000", 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1339999999999998e-06 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971285+0000", 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.971294+0000", 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6840000000000008e-06 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972838+0000", 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001543981 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.972887+0000", 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9013e-05 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:399 3.0s0 3:00c2d3b1:::smithi10238019-29:head [copy-from ver 2] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.95794684600000002, 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01110614, 2024-09-17T17:30:02.411 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 399 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:02.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343265+0000", 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343273+0000", 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343291+0000", 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7764000000000001e-05 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343299+0000", 2024-09-17T17:30:02.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1629999999999996e-06 2024-09-17T17:30:02.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343366+0000", 2024-09-17T17:30:02.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6398999999999999e-05 2024-09-17T17:30:02.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343443+0000", 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7731000000000005e-05 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.352245+0000", 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088010680000000004 2024-09-17T17:30:02.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353574+0000", 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013293109999999999 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.354245+0000", 2024-09-17T17:30:02.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067153099999999995 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.354369+0000", 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012345600000000001 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:400 3.0s0 3:00c2d3b1:::smithi10238019-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.95789143499999996, 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010156996, 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 400 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343319+0000", 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343321+0000", 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5689999999999999e-06 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343335+0000", 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3541e-05 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343343+0000", 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2740000000000007e-06 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343584+0000", 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241179 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343618+0000", 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4147999999999999e-05 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353248+0000", 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096298000000000009 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:02.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353357+0000", 2024-09-17T17:30:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109281 2024-09-17T17:30:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353475+0000", 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011772 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 50/35 ECSubWrite(tid=89, reqid=client.4566.0:401, at_version=50'26, trim_to=0'0, roll_forward_to=49'24))", 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.353761+0000", 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.94744851299999999, 2024-09-17T17:30:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021311580000000002, 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353761+0000", 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353761+0000", 2024-09-17T17:30:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353762+0000", 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.06e-07 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353781+0000", 2024-09-17T17:30:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9221000000000001e-05 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353811+0000", 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0348000000000001e-05 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353825+0000", 2024-09-17T17:30:02.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3859000000000001e-05 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353890+0000", 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4774e-05 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353898+0000", 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355860+0000", 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019618719999999999 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355892+0000", 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1884000000000002e-05 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 50/35 ECSubWrite(tid=90, reqid=client.4566.0:402, at_version=50'27, trim_to=0'0, roll_forward_to=49'24))", 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.355044+0000", 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.946165335, 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001851143, 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:02.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355044+0000", 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355044+0000", 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355045+0000", 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4300000000000002e-07 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355061+0000", 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6027000000000001e-05 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355093+0000", 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2393e-05 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355101+0000", 2024-09-17T17:30:02.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1749999999999999e-06 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355383+0000", 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028268599999999998 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.355394+0000", 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0314e-05 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356861+0000", 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001467559 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356895+0000", 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4146000000000001e-05 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:02.439 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:02.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T17:30:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:30:02.444 INFO:tasks.rados.rados.0.smithi102.stdout:152: finishing copy_from racing read to smithi10238019-29 2024-09-17T17:30:02.444 INFO:tasks.rados.rados.0.smithi102.stdout:152: finishing copy_from to smithi10238019-29 2024-09-17T17:30:02.444 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 29 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-17T17:30:02.444 INFO:tasks.rados.rados.0.smithi102.stdout:151: finishing write tid 3 to smithi10238019-18 2024-09-17T17:30:02.444 INFO:tasks.rados.rados.0.smithi102.stdout:151: finishing write tid 4 to smithi10238019-18 2024-09-17T17:30:02.444 INFO:tasks.rados.rados.0.smithi102.stdout:153: finishing rollback tid 0 to smithi10238019-7 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: finishing write tid 5 to smithi10238019-18 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: oid 18 updating version 0 to 37 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: oid 18 version 37 is already newer than 35 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: oid 18 version 37 is already newer than 36 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: oid 18 version 37 is already newer than 34 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 18 v 37 (ObjNum 83 snap 10 seq_num 83) dirty exists 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: left oid 18 (ObjNum 83 snap 10 seq_num 83) 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:153: finishing rollback tid 1 to smithi10238019-7 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:153: finishing rollback tid 2 to smithi10238019-7 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 28 (ObjNum 76 snap 4 seq_num 76) dirty exists 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:151: done (3 left) 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:152: done (2 left) 2024-09-17T17:30:02.445 INFO:tasks.rados.rados.0.smithi102.stdout:153: done (1 left) 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:154: done (0 left) 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:155: rollback oid 16 current snap is 10 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 16 to 7 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:156: read oid 42 snap -1 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:156: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:157: copy_from oid 5 from oid 36 current snap is 10 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 37 current snap is 10 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:158: seq_num 84 ranges {1736704=655360,2392064=638976,3031040=131072} 2024-09-17T17:30:02.446 INFO:tasks.rados.rados.0.smithi102.stdout:158: writing smithi10238019-37 from 1736704 to 2392064 tid 1 2024-09-17T17:30:02.447 INFO:tasks.rados.rados.0.smithi102.stdout:158: writing smithi10238019-37 from 2392064 to 3031040 tid 2 2024-09-17T17:30:02.447 INFO:tasks.rados.rados.0.smithi102.stdout:158: writing smithi10238019-37 from 3031040 to 3162112 tid 3 2024-09-17T17:30:02.447 INFO:tasks.rados.rados.0.smithi102.stdout:159: snap_remove snap 6 2024-09-17T17:30:02.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T17:30:02.448 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:02.439+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:02.542 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:02.540+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:02.567 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:02.567 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:02.567 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:02.567 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:02.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:02.643 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:02.641+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:02.684 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:02.684 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:02.685 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:02.685 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:02.685 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:02.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:02.744 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:02.742+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 14.322405902, 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:30:02.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:179 3.4s0 3:26d49efd:::smithi10238019-44:head [delete] snapc 2=[2] ondisk+write+known_if_redirected+full_force+supports_pool_eio e37)", 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:30:02.807 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 13.809011289000001, 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039941960000000002, 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 179 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990178+0000", 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990179+0000", 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.22e-06 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990184+0000", 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1239999999999998e-06 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990194+0000", 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0377000000000001e-05 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990200+0000", 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5690000000000001e-06 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990248+0000", 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7615999999999999e-05 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.990300+0000", 2024-09-17T17:30:02.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2364000000000002e-05 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991074+0000", 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077367199999999999 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.992851+0000", 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001776632 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994099+0000", 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012486610000000001 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994172+0000", 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2960999999999995e-05 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 13.804139664000001, 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:30:02.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.112422141, 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:203 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~557056] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.09522123, 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0069058180000000002, 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 203 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.822 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703971+0000", 2024-09-17T17:30:02.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4849999999999999e-06 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704582+0000", 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00061129700000000001 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704592+0000", 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0094e-05 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704598+0000", 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6019999999999999e-06 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704655+0000", 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7107000000000001e-05 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704712+0000", 2024-09-17T17:30:02.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7435000000000002e-05 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.705382+0000", 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067020899999999995 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706343+0000", 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00096062299999999997 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710818+0000", 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044745519999999997 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710874+0000", 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6413999999999999e-05 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.827 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.092978859, 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:30:02.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:30:02.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:30:02.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:02.833 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.092591956, 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:30:02.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:30:02.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:30:02.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.839 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.092558588999999, 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:30:02.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:02.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.088109118, 2024-09-17T17:30:02.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:02.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:30:02.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s6 39/35 ECSubWrite(tid=29, reqid=client.4566.0:210, at_version=39'7, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.713548+0000", 2024-09-17T17:30:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.085641198999999, 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033443930000000002, 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713548+0000", 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713548+0000", 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713549+0000", 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0899999999999997e-07 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713566+0000", 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7529000000000001e-05 2024-09-17T17:30:02.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715774+0000", 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002207741 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715780+0000", 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9699999999999996e-06 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716322+0000", 2024-09-17T17:30:02.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054210100000000002 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716326+0000", 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3229999999999999e-06 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716874+0000", 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054769900000000002 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716893+0000", 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8720999999999999e-05 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s6 39/35 ECSubWrite(tid=31, reqid=client.4566.0:211, at_version=39'8, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.713568+0000", 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.085621099000001, 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003603948, 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713568+0000", 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713568+0000", 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713569+0000", 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0899999999999997e-07 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.713578+0000", 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8419999999999994e-06 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715796+0000", 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022186279999999998 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.715801+0000", 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8289999999999998e-06 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716515+0000", 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000713676 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716520+0000", 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2020000000000003e-06 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.717153+0000", 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00063337699999999996 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.717172+0000", 2024-09-17T17:30:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9085000000000001e-05 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:367 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~589824] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4604983030000001, 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012183095999999999, 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 367 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338693+0000", 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.795e-06 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341609+0000", 2024-09-17T17:30:02.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029156210000000002 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341623+0000", 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4583000000000001e-05 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341630+0000", 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2230000000000001e-06 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341691+0000", 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0878000000000002e-05 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341771+0000", 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9308999999999996e-05 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343069+0000", 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012979420000000001 2024-09-17T17:30:02.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348577+0000", 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0055080229999999999 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350781+0000", 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002204742 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350874+0000", 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2979999999999994e-05 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:368 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~573440] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4568582860000001, 2024-09-17T17:30:02.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0088382370000000005, 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 368 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:02.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342332+0000", 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.08e-06 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342959+0000", 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062686600000000001 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343062+0000", 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010252 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343083+0000", 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1135e-05 2024-09-17T17:30:02.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344260+0000", 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011769969999999999 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344389+0000", 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012922399999999999 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345712+0000", 2024-09-17T17:30:02.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001323182 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348773+0000", 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030611649999999998 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351091+0000", 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023171590000000001 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351169+0000", 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8909e-05 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:369 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~196608] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4557565769999998, 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0079413130000000002, 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 369 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343434+0000", 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4699999999999999e-06 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343667+0000", 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023281800000000001 2024-09-17T17:30:02.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343680+0000", 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2524000000000001e-05 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343686+0000", 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.19e-06 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346596+0000", 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002909867 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346803+0000", 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00020759199999999999 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348042+0000", 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001238636 2024-09-17T17:30:02.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350047+0000", 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002004774 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351299+0000", 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001252364 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351374+0000", 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5078000000000002e-05 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:370 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4554888340000001, 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0082819799999999996, 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 370 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3799999999999997e-07 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343710+0000", 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2570000000000007e-06 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343715+0000", 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6829999999999996e-06 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343721+0000", 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:02.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349122+0000", 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005400841 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349229+0000", 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107076 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349735+0000", 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050621599999999998 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351781+0000", 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020453279999999999 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351908+0000", 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012790499999999999 2024-09-17T17:30:02.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351983+0000", 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4154999999999997e-05 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:371 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:02.883 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4554617909999998, 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010270708, 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 371 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4799999999999999e-07 2024-09-17T17:30:02.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343730+0000", 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4389999999999999e-06 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343735+0000", 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1919999999999998e-06 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343738+0000", 2024-09-17T17:30:02.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1360000000000001e-06 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352473+0000", 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0087356429999999995 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352570+0000", 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6261999999999994e-05 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353998+0000", 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001428688 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 49/35 ECSubWriteReply(tid=20, last_complete=49'13, committed=1, applied=1))", 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.344258+0000", 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4549314199999999, 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0042485860000000004, 2024-09-17T17:30:02.888 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344258+0000", 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344258+0000", 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344259+0000", 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.263e-06 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344266+0000", 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5640000000000002e-06 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344289+0000", 2024-09-17T17:30:02.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2989999999999998e-05 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344298+0000", 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1400000000000006e-06 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348477+0000", 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0041787960000000002 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348507+0000", 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9833e-05 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s2 49/35 ECSubWrite(tid=39, reqid=client.4566.0:365, at_version=49'28, trim_to=0'0, roll_forward_to=48'26))", 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.345060+0000", 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.4541294570000001, 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00375194, 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345060+0000", 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345060+0000", 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.895 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345061+0000", 2024-09-17T17:30:02.895 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7600000000000006e-07 2024-09-17T17:30:02.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.895 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345569+0000", 2024-09-17T17:30:02.896 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050776100000000002 2024-09-17T17:30:02.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.896 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345647+0000", 2024-09-17T17:30:02.897 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8164000000000006e-05 2024-09-17T17:30:02.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.897 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.897 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345667+0000", 2024-09-17T17:30:02.898 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0324000000000001e-05 2024-09-17T17:30:02.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.900 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.900 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345765+0000", 2024-09-17T17:30:02.900 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7532000000000001e-05 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345779+0000", 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4253000000000001e-05 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348778+0000", 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029994000000000002 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.902 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348812+0000", 2024-09-17T17:30:02.903 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3529999999999999e-05 2024-09-17T17:30:02.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.6s4 51/35 ECSubWrite(tid=121, reqid=osd.13.0:121, at_version=51'51, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.342098+0000", 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.457091519, 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0034192509999999999, 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342098+0000", 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342098+0000", 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342098+0000", 2024-09-17T17:30:02.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0699999999999998e-07 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342125+0000", 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6352000000000001e-05 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342454+0000", 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00032948599999999998 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342464+0000", 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9559999999999992e-06 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342535+0000", 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0755999999999998e-05 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342546+0000", 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0698e-05 2024-09-17T17:30:02.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.345447+0000", 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029012740000000001 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.345517+0000", 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0321999999999995e-05 2024-09-17T17:30:02.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.909 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.909 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.909 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.909 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 51/32 ECSubWrite(tid=95, reqid=osd.11.0:95, at_version=51'35, trim_to=0'0, roll_forward_to=48'33))", 2024-09-17T17:30:02.909 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.344344+0000", 2024-09-17T17:30:02.910 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.45484552299999997, 2024-09-17T17:30:02.910 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0036358699999999998, 2024-09-17T17:30:02.910 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:02.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:02.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:02.910 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:02.911 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.911 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:02.911 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344344+0000", 2024-09-17T17:30:02.911 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.912 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.912 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:02.912 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344344+0000", 2024-09-17T17:30:02.912 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:02.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344344+0000", 2024-09-17T17:30:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.15e-07 2024-09-17T17:30:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344367+0000", 2024-09-17T17:30:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2728999999999999e-05 2024-09-17T17:30:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.915 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:02.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344468+0000", 2024-09-17T17:30:02.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010147 2024-09-17T17:30:02.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.916 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:02.916 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344476+0000", 2024-09-17T17:30:02.916 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0910000000000004e-06 2024-09-17T17:30:02.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.916 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:02.917 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.345454+0000", 2024-09-17T17:30:02.917 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00097821400000000004 2024-09-17T17:30:02.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.917 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:02.918 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.345462+0000", 2024-09-17T17:30:02.918 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3520000000000007e-06 2024-09-17T17:30:02.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.918 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:02.919 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.347840+0000", 2024-09-17T17:30:02.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023780619999999998 2024-09-17T17:30:02.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:02.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:02.919 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.347980+0000", 2024-09-17T17:30:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013963699999999999 2024-09-17T17:30:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:02.921 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:02.921 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:02.923 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:02.921+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:02.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:03.024 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:03.022+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:03.073 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:03.073 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:03.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:03.125 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:03.124+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:03.188 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:03.188 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:03.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:03.226 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:03.224+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 14.310615781999999, 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:30:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.684789+0000", 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.617592862, 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015238630000000001, 2024-09-17T17:30:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684789+0000", 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684789+0000", 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684790+0000", 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7399999999999991e-07 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684817+0000", 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7800000000000001e-05 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684856+0000", 2024-09-17T17:30:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8268e-05 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684874+0000", 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8845e-05 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.685099+0000", 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000224434 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8009999999999998e-06 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686110+0000", 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001001395 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686312+0000", 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000202346 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 39/35 ECSubWrite(tid=20, reqid=client.4566.0:191, at_version=39'17, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.689581+0000", 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.612800306, 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001530132, 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689581+0000", 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689581+0000", 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689582+0000", 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6700000000000003e-07 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689753+0000", 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017074899999999999 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689784+0000", 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1124000000000002e-05 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689790+0000", 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7959999999999998e-06 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689845+0000", 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4627999999999999e-05 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.689856+0000", 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0678e-05 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.691077+0000", 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012208570000000001 2024-09-17T17:30:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.691111+0000", 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4632999999999999e-05 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.581679477, 2024-09-17T17:30:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002528565, 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720703+0000", 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5300000000000004e-07 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721318+0000", 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000615779 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721351+0000", 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2486e-05 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721360+0000", 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8729999999999996e-06 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721434+0000", 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4011999999999995e-05 2024-09-17T17:30:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721445+0000", 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0903999999999999e-05 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723198+0000", 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001753641 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723231+0000", 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2217e-05 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 39/35 ECSubWrite(tid=23, reqid=client.4566.0:220, at_version=39'18, trim_to=0'0, roll_forward_to=39'17))", 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.722057+0000", 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.580324055, 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00151669, 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722057+0000", 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722057+0000", 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722058+0000", 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0599999999999999e-07 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722107+0000", 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9092999999999999e-05 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722141+0000", 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3643000000000003e-05 2024-09-17T17:30:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722149+0000", 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9459999999999998e-06 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722259+0000", 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010987 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722269+0000", 2024-09-17T17:30:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0155e-05 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723541+0000", 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001272256 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723574+0000", 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2920999999999999e-05 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.5765427420000009, 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:30:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.692888+0000", 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.6094930289999994, 2024-09-17T17:30:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001928377, 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692888+0000", 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692888+0000", 2024-09-17T17:30:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692889+0000", 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5200000000000002e-07 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692945+0000", 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5787e-05 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692999+0000", 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4858999999999998e-05 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693011+0000", 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1496e-05 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693138+0000", 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000126744 2024-09-17T17:30:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693148+0000", 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0583e-05 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694769+0000", 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00162119 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694817+0000", 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7265999999999998e-05 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=37, reqid=osd.8.0:37, at_version=43'27, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.6093032520000001, 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022426339999999999, 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693078+0000", 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9400000000000001e-07 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693102+0000", 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3547e-05 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693145+0000", 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3263000000000002e-05 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693153+0000", 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6690000000000006e-06 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693240+0000", 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7158999999999997e-05 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693250+0000", 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695273+0000", 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020229200000000001 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695321+0000", 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8044000000000003e-05 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.6085148700000005, 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:30:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.6082727600000002, 2024-09-17T17:30:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:30:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.700980+0000", 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.6014009619999996, 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020660069999999999, 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700980+0000", 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700980+0000", 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700981+0000", 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1399999999999996e-07 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.700995+0000", 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3764000000000001e-05 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701031+0000", 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5837000000000003e-05 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701040+0000", 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701116+0000", 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6530000000000001e-05 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701126+0000", 2024-09-17T17:30:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5990000000000008e-06 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703000+0000", 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001874173 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703046+0000", 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6338999999999998e-05 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 43/32 ECSubWrite(tid=55, reqid=osd.11.0:55, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.6007494930000004, 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002034059, 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:30:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701632+0000", 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701655+0000", 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2809000000000001e-05 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701737+0000", 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2349999999999996e-05 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701745+0000", 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701985+0000", 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023929299999999999 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701996+0000", 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.118e-05 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703604+0000", 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001608682 2024-09-17T17:30:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703666+0000", 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1535e-05 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 49/35 ECSubWrite(tid=39, reqid=client.4566.0:365, at_version=49'28, trim_to=0'0, roll_forward_to=48'26))", 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338590+0000", 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.9637909950000001, 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019648769999999999, 2024-09-17T17:30:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338590+0000", 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338590+0000", 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338591+0000", 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.13e-07 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338843+0000", 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00025211199999999997 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338891+0000", 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7762000000000001e-05 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338908+0000", 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6918e-05 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.339007+0000", 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9097000000000003e-05 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.339021+0000", 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4335000000000001e-05 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.340509+0000", 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001487602 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.340555+0000", 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6437999999999998e-05 2024-09-17T17:30:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 49/32 ECSubWrite(tid=27, reqid=client.4566.0:372, at_version=49'20, trim_to=0'0, roll_forward_to=48'18))", 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.350379+0000", 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.9520027500000001, 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019631150000000001, 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350379+0000", 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350379+0000", 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350379+0000", 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.41e-07 2024-09-17T17:30:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350835+0000", 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00045573100000000002 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350882+0000", 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6882000000000003e-05 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350890+0000", 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5259999999999993e-06 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350977+0000", 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7100000000000003e-05 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350988+0000", 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0688e-05 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352299+0000", 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013113930000000001 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352342+0000", 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2553999999999999e-05 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 49/32 ECSubWrite(tid=29, reqid=client.4566.0:373, at_version=49'21, trim_to=0'0, roll_forward_to=49'20))", 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.353979+0000", 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.9484028840000001, 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021980630000000001, 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353979+0000", 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353979+0000", 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353979+0000", 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2399999999999997e-07 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.354734+0000", 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00075502000000000002 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.354780+0000", 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6032000000000002e-05 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.354790+0000", 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8970000000000003e-06 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.354881+0000", 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0799999999999998e-05 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.354897+0000", 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5486999999999999e-05 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.356136+0000", 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012398909999999999 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.356177+0000", 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0111999999999998e-05 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 51/32 ECSubWrite(tid=34, reqid=osd.7.0:34, at_version=51'25, trim_to=0'0, roll_forward_to=49'23))", 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.336195+0000", 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.96618656400000003, 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020528170000000002, 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336195+0000", 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336195+0000", 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336195+0000", 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6400000000000001e-07 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336222+0000", 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6985e-05 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336273+0000", 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1196000000000002e-05 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336285+0000", 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1438e-05 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336364+0000", 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9672000000000006e-05 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336382+0000", 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7799999999999999e-05 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.338200+0000", 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001817936 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.338248+0000", 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7626000000000001e-05 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 51/32 ECSubWrite(tid=35, reqid=osd.7.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=51'25))", 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.339732+0000", 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.96264894700000003, 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018952870000000001, 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339732+0000", 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339732+0000", 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339734+0000", 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.17e-06 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339753+0000", 2024-09-17T17:30:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9159000000000001e-05 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339803+0000", 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0689999999999997e-05 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339814+0000", 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0501e-05 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339886+0000", 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1734000000000001e-05 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339897+0000", 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.096e-05 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.341586+0000", 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001689086 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.341628+0000", 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1987000000000003e-05 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 51/35 ECSubWrite(tid=41, reqid=osd.10.0:41, at_version=51'30, trim_to=0'0, roll_forward_to=49'28))", 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.340280+0000", 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.96210167400000002, 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019954980000000001, 2024-09-17T17:30:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340280+0000", 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340280+0000", 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340280+0000", 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3999999999999998e-07 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340308+0000", 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7611e-05 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340366+0000", 2024-09-17T17:30:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8051999999999999e-05 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340414+0000", 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8751000000000001e-05 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340480+0000", 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5843e-05 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.340492+0000", 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2052000000000001e-05 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342234+0000", 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001742171 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.342275+0000", 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0778000000000002e-05 2024-09-17T17:30:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 51/32 ECSubWrite(tid=36, reqid=client.4566.0:412, at_version=51'27, trim_to=0'0, roll_forward_to=51'25))", 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.358526+0000", 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.94385497299999999, 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002044662, 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358526+0000", 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358526+0000", 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358528+0000", 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.72e-06 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358999+0000", 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00047050899999999998 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359042+0000", 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3618999999999997e-05 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359056+0000", 2024-09-17T17:30:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3542e-05 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359117+0000", 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0680000000000002e-05 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359123+0000", 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7660000000000001e-06 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.360528+0000", 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001404621 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.360571+0000", 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3204999999999997e-05 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 51/32 ECSubWrite(tid=37, reqid=client.4566.0:413, at_version=51'28, trim_to=0'0, roll_forward_to=51'25))", 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.362636+0000", 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.93974559800000002, 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017365429999999999, 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.362636+0000", 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.362636+0000", 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.362637+0000", 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0999999999999997e-07 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.363209+0000", 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057258700000000005 2024-09-17T17:30:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.363252+0000", 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3031999999999997e-05 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.363262+0000", 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7370000000000001e-06 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.363332+0000", 2024-09-17T17:30:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9554000000000005e-05 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.363341+0000", 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.364343+0000", 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010011849999999999 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.364372+0000", 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9799999999999999e-05 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.400 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T17:30:03.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:03.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:159: done (4 left) 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 39 current snap is 10 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:155: finishing rollback tid 1 to smithi10238019-16 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:155: finishing rollback tid 2 to smithi10238019-16 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 30 (ObjNum 76 snap 4 seq_num 76) dirty exists 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:157: finishing copy_from racing read to smithi10238019-5 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:156: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-17T17:30:03.406 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 53220 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:157: finishing copy_from to smithi10238019-5 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 5 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: finishing write tid 1 to smithi10238019-37 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: finishing write tid 2 to smithi10238019-37 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: finishing write tid 3 to smithi10238019-37 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: finishing write tid 4 to smithi10238019-37 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: finishing write tid 5 to smithi10238019-37 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: oid 37 updating version 0 to 30 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: oid 37 version 30 is already newer than 29 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: oid 37 version 30 is already newer than 28 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:158: oid 37 version 30 is already newer than 27 2024-09-17T17:30:03.407 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 37 v 30 (ObjNum 84 snap 10 seq_num 84) dirty exists 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:158: left oid 37 (ObjNum 84 snap 10 seq_num 84) 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:160: seq_num 85 ranges {638976=671744,1310720=770048} 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:160: writing smithi10238019-39 from 638976 to 1310720 tid 1 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:160: writing smithi10238019-39 from 1310720 to 2080768 tid 2 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:155: done (4 left) 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:156: done (3 left) 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:157: done (2 left) 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:158: done (1 left) 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:161: read oid 40 snap -1 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:161: expect (ObjNum 60 snap 2 seq_num 60) 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 34 current snap is 10 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:162: seq_num 86 ranges {278528=688128,966656=540672,1507328=229376} 2024-09-17T17:30:03.408 INFO:tasks.rados.rados.0.smithi102.stdout:162: writing smithi10238019-34 from 278528 to 966656 tid 1 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:162: writing smithi10238019-34 from 966656 to 1507328 tid 2 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:162: writing smithi10238019-34 from 1507328 to 1736704 tid 3 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:163: rollback oid 20 current snap is 10 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 20 to 9 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: finishing write tid 1 to smithi10238019-39 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: finishing write tid 2 to smithi10238019-39 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: finishing write tid 3 to smithi10238019-39 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:164: read oid 11 snap -1 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:164: expect deleted 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:165: copy_from oid 43 from oid 17 current snap is 10 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: finishing write tid 4 to smithi10238019-39 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: oid 39 updating version 0 to 27 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: oid 39 version 27 is already newer than 26 2024-09-17T17:30:03.409 INFO:tasks.rados.rados.0.smithi102.stdout:160: oid 39 updating version 27 to 28 2024-09-17T17:30:03.410 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 39 v 28 (ObjNum 85 snap 10 seq_num 85) dirty exists 2024-09-17T17:30:03.410 INFO:tasks.rados.rados.0.smithi102.stdout:160: left oid 39 (ObjNum 85 snap 10 seq_num 85) 2024-09-17T17:30:03.410 INFO:tasks.rados.rados.0.smithi102.stdout:160: done (5 left) 2024-09-17T17:30:03.410 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 27 current snap is 10 2024-09-17T17:30:03.410 INFO:tasks.rados.rados.0.smithi102.stdout:166: seq_num 87 ranges {311296=704512,1015808=688128,1703936=81920} 2024-09-17T17:30:03.410 INFO:tasks.rados.rados.0.smithi102.stdout:166: writing smithi10238019-27 from 311296 to 1015808 tid 1 2024-09-17T17:30:03.410 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:03.399+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:03.411 INFO:tasks.rados.rados.0.smithi102.stdout:166: writing smithi10238019-27 from 1015808 to 1703936 tid 2 2024-09-17T17:30:03.412 INFO:tasks.rados.rados.0.smithi102.stdout:166: writing smithi10238019-27 from 1703936 to 1785856 tid 3 2024-09-17T17:30:03.412 INFO:tasks.rados.rados.0.smithi102.stdout:167: delete oid 36 current snap is 10 2024-09-17T17:30:03.412 INFO:tasks.rados.rados.0.smithi102.stdout:162: finishing write tid 1 to smithi10238019-34 2024-09-17T17:30:03.412 INFO:tasks.rados.rados.0.smithi102.stdout:161: expect (ObjNum 60 snap 2 seq_num 60) 2024-09-17T17:30:03.412 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:03.426 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:03.444 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:03.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:163: finishing rollback tid 0 to smithi10238019-20 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:162: finishing write tid 2 to smithi10238019-34 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:163: finishing rollback tid 1 to smithi10238019-20 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:162: finishing write tid 3 to smithi10238019-34 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:163: finishing rollback tid 2 to smithi10238019-20 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 20 v 22 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:162: finishing write tid 4 to smithi10238019-34 2024-09-17T17:30:03.464 INFO:tasks.rados.rados.0.smithi102.stdout:165: finishing copy_from racing read to smithi10238019-43 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:161: done (6 left) 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:163: done (5 left) 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:164: done (4 left) 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:167: done (3 left) 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 8 current snap is 10 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:166: finishing write tid 1 to smithi10238019-27 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:165: finishing copy_from to smithi10238019-43 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 43 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:162: finishing write tid 5 to smithi10238019-34 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:162: oid 34 updating version 0 to 39 2024-09-17T17:30:03.465 INFO:tasks.rados.rados.0.smithi102.stdout:162: oid 34 updating version 39 to 42 2024-09-17T17:30:03.466 INFO:tasks.rados.rados.0.smithi102.stdout:162: oid 34 version 42 is already newer than 40 2024-09-17T17:30:03.466 INFO:tasks.rados.rados.0.smithi102.stdout:162: oid 34 version 42 is already newer than 41 2024-09-17T17:30:03.466 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 34 v 42 (ObjNum 86 snap 10 seq_num 86) dirty exists 2024-09-17T17:30:03.466 INFO:tasks.rados.rados.0.smithi102.stdout:162: left oid 34 (ObjNum 86 snap 10 seq_num 86) 2024-09-17T17:30:03.466 INFO:tasks.rados.rados.0.smithi102.stdout:166: finishing write tid 2 to smithi10238019-27 2024-09-17T17:30:03.466 INFO:tasks.rados.rados.0.smithi102.stdout:168: seq_num 89 ranges {131072=737280,868352=573440,1441792=196608} 2024-09-17T17:30:03.468 INFO:tasks.rados.rados.0.smithi102.stdout:168: writing smithi10238019-8 from 131072 to 868352 tid 1 2024-09-17T17:30:03.471 INFO:tasks.rados.rados.0.smithi102.stdout:168: writing smithi10238019-8 from 868352 to 1441792 tid 2 2024-09-17T17:30:03.473 INFO:tasks.rados.rados.0.smithi102.stdout:168: writing smithi10238019-8 from 1441792 to 1638400 tid 3 2024-09-17T17:30:03.473 INFO:tasks.rados.rados.0.smithi102.stdout:162: done (3 left) 2024-09-17T17:30:03.473 INFO:tasks.rados.rados.0.smithi102.stdout:165: done (2 left) 2024-09-17T17:30:03.473 INFO:tasks.rados.rados.0.smithi102.stdout:169: copy_from oid 33 from oid 21 current snap is 10 2024-09-17T17:30:03.473 INFO:tasks.rados.rados.0.smithi102.stdout:170: snap_remove snap 4 2024-09-17T17:30:03.502 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:03.500+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:03.521 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:03.522 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:03.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:03.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:03.603 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:03.601+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:03.610 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:03.610 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:03.632 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:03.637 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:03.637 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:03.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:03.653 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:03.673 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:03.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:03.704 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:03.702+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:03.715 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:03.715 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:03.754 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s2 37/35 ECSubWrite(tid=8, reqid=client.4566.0:179, at_version=37'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 14.76036869, 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021580079999999999, 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991038+0000", 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6e-07 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991063+0000", 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4984999999999999e-05 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991151+0000", 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7474999999999999e-05 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991160+0000", 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2569999999999996e-06 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991232+0000", 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1891999999999995e-05 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991243+0000", 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1243999999999999e-05 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993161+0000", 2024-09-17T17:30:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019180689999999999 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993196+0000", 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4826000000000002e-05 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/35 ECSubWrite(tid=10, reqid=osd.1.0:10, at_version=39'8, trim_to=0'0, roll_forward_to=37'6))", 2024-09-17T17:30:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.681750+0000", 2024-09-17T17:30:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 13.069656373000001, 2024-09-17T17:30:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001759785, 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681750+0000", 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681750+0000", 2024-09-17T17:30:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681750+0000", 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681775+0000", 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4624999999999999e-05 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681830+0000", 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4746000000000001e-05 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681841+0000", 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0659000000000001e-05 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681919+0000", 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8812000000000004e-05 2024-09-17T17:30:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681938+0000", 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9059000000000002e-05 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683480+0000", 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015416850000000001 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683510+0000", 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9944e-05 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.058488359, 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025590249999999999, 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1799999999999999e-07 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692948+0000", 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9323999999999999e-05 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693000+0000", 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1894999999999997e-05 2024-09-17T17:30:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693011+0000", 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1481000000000001e-05 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693101+0000", 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0237999999999999e-05 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693114+0000", 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2325e-05 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695440+0000", 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023263030000000001 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695477+0000", 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7240999999999998e-05 2024-09-17T17:30:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.0573462070000001, 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026508170000000002, 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694061+0000", 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5099999999999998e-07 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694087+0000", 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5571999999999998e-05 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694133+0000", 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5583e-05 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694142+0000", 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6500000000000008e-06 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694227+0000", 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5159000000000002e-05 2024-09-17T17:30:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694246+0000", 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8723e-05 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696667+0000", 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024210709999999999 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696711+0000", 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4107999999999998e-05 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.701581+0000", 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.0498254159999991, 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021540719999999999, 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701581+0000", 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701581+0000", 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701582+0000", 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.164e-06 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701600+0000", 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7948000000000002e-05 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701713+0000", 2024-09-17T17:30:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000113045 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701723+0000", 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7550000000000006e-06 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.702309+0000", 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058631999999999998 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.702320+0000", 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0322e-05 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703695+0000", 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013753560000000001 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703735+0000", 2024-09-17T17:30:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0161999999999999e-05 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 43/32 ECSubWrite(tid=55, reqid=osd.11.0:55, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.701664+0000", 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.0497426740000009, 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0016259239999999999, 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701664+0000", 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701664+0000", 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701664+0000", 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9499999999999998e-07 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701691+0000", 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6565000000000001e-05 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701765+0000", 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4313999999999994e-05 2024-09-17T17:30:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701787+0000", 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2010000000000001e-05 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701875+0000", 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8344000000000007e-05 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.701887+0000", 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1345000000000001e-05 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703231+0000", 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001343994 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.703290+0000", 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9057000000000001e-05 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 50/35 ECSubWrite(tid=58, reqid=client.4566.0:399, at_version=50'14, trim_to=0'0, roll_forward_to=40'12))", 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:01.352250+0000", 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.399156208, 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001756688, 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352250+0000", 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352250+0000", 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352251+0000", 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.11e-07 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352416+0000", 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000165081 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352459+0000", 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3115999999999999e-05 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352470+0000", 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1440999999999999e-05 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352560+0000", 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9767999999999998e-05 2024-09-17T17:30:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.352568+0000", 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2209999999999997e-06 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.353953+0000", 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013851239999999999 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:01.354007+0000", 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3826000000000003e-05 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 51/35 ECSubWrite(tid=60, reqid=osd.12.0:60, at_version=51'16, trim_to=0'0, roll_forward_to=50'14))", 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.336139+0000", 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.4152671589999999, 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024531190000000001, 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336139+0000", 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336139+0000", 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336140+0000", 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.31e-07 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336167+0000", 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7548000000000001e-05 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336270+0000", 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000103272 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336283+0000", 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2427e-05 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336359+0000", 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6268999999999998e-05 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336371+0000", 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1610000000000001e-05 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.338555+0000", 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021840760000000001 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.338593+0000", 2024-09-17T17:30:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7786e-05 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/35 ECSubWrite(tid=61, reqid=osd.12.0:61, at_version=51'30, trim_to=0'0, roll_forward_to=48'28))", 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.336169+0000", 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.4152374700000001, 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002418828, 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336169+0000", 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336169+0000", 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336169+0000", 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3999999999999998e-07 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336215+0000", 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5515e-05 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336320+0000", 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000105499 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336328+0000", 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4070000000000004e-06 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336403+0000", 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5699999999999997e-05 2024-09-17T17:30:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336416+0000", 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2556e-05 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.338540+0000", 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021236409999999999 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.338588+0000", 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8269999999999997e-05 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/32 ECSubWrite(tid=34, reqid=osd.7.0:34, at_version=51'25, trim_to=0'0, roll_forward_to=49'23))", 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.336230+0000", 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.4151767450000001, 2024-09-17T17:30:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017488619999999999, 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336230+0000", 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336230+0000", 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336230+0000", 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7700000000000001e-07 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336256+0000", 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5579999999999999e-05 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336303+0000", 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7148000000000003e-05 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336645+0000", 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000341944 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336725+0000", 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0654000000000004e-05 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.336736+0000", 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0691e-05 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.337915+0000", 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001178873 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.337979+0000", 2024-09-17T17:30:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3794999999999995e-05 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/32 ECSubWrite(tid=35, reqid=osd.7.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=51'25))", 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.339762+0000", 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.4116450650000001, 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015439010000000001, 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339762+0000", 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339762+0000", 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339763+0000", 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9300000000000006e-07 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339780+0000", 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7167999999999999e-05 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339827+0000", 2024-09-17T17:30:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7055999999999998e-05 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339836+0000", 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4620000000000002e-06 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339909+0000", 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2482000000000002e-05 2024-09-17T17:30:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.339918+0000", 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6229999999999992e-06 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.341243+0000", 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001324537 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.341305+0000", 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2580000000000001e-05 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 51/32 ECSubWrite(tid=95, reqid=osd.11.0:95, at_version=51'35, trim_to=0'0, roll_forward_to=48'33))", 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.343992+0000", 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.4074145439999999, 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002212641, 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.343992+0000", 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.343992+0000", 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.343992+0000", 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.65e-07 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344020+0000", 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7685999999999999e-05 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344072+0000", 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2240000000000001e-05 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344082+0000", 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6919999999999997e-06 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344182+0000", 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9562999999999995e-05 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344193+0000", 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1044e-05 2024-09-17T17:30:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.346161+0000", 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019683600000000002 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.346205+0000", 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3690999999999999e-05 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/35 ECSubWrite(tid=62, reqid=osd.12.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.344539+0000", 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.4068678320000001, 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017477, 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344539+0000", 2024-09-17T17:30:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344539+0000", 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344540+0000", 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4200000000000005e-07 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344552+0000", 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2761e-05 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344606+0000", 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3560999999999998e-05 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344615+0000", 2024-09-17T17:30:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0659999999999993e-06 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344690+0000", 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5111000000000006e-05 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344701+0000", 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.045e-05 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.346249+0000", 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001547989 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.346286+0000", 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7920000000000003e-05 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 51/35 ECSubWrite(tid=63, reqid=osd.12.0:60, at_version=0'0, trim_to=0'0, roll_forward_to=51'16))", 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.344557+0000", 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.406849816, 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0016994429999999999, 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344557+0000", 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344557+0000", 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344557+0000", 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7500000000000001e-07 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344563+0000", 2024-09-17T17:30:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8590000000000001e-06 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344883+0000", 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00032032099999999998 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344892+0000", 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9090000000000006e-06 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344974+0000", 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2207999999999996e-05 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.344984+0000", 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4390000000000006e-06 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.346221+0000", 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012374160000000001 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.346256+0000", 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5015999999999998e-05 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/35 ECSubWrite(tid=65, reqid=client.4566.0:410, at_version=51'31, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.349748+0000", 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.401658458, 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018728130000000001, 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.349748+0000", 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.349748+0000", 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.349749+0000", 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5099999999999998e-07 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.349947+0000", 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000197612 2024-09-17T17:30:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.350006+0000", 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9302999999999998e-05 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.350015+0000", 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1290000000000002e-06 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.350084+0000", 2024-09-17T17:30:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8483000000000001e-05 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.350093+0000", 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9840000000000007e-06 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.351578+0000", 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001485283 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.351621+0000", 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3068000000000002e-05 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/32 ECSubWrite(tid=36, reqid=client.4566.0:412, at_version=51'27, trim_to=0'0, roll_forward_to=51'25))", 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:02.358673+0000", 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.392733556, 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022703340000000002, 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358673+0000", 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358673+0000", 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.358674+0000", 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359327+0000", 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00065276899999999998 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359367+0000", 2024-09-17T17:30:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9882000000000002e-05 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359376+0000", 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1900000000000001e-06 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359446+0000", 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0557999999999998e-05 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.359458+0000", 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1846e-05 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.360895+0000", 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001436657 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:02.360943+0000", 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8513999999999997e-05 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.361768429, 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026689740000000002, 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389639+0000", 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.06e-07 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390292+0000", 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00065341900000000003 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390322+0000", 2024-09-17T17:30:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.921e-05 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390328+0000", 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5599999999999999e-06 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390385+0000", 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6484999999999999e-05 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390396+0000", 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1409e-05 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392271+0000", 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001875383 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392307+0000", 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5802e-05 2024-09-17T17:30:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 52/32 ECSubWrite(tid=106, reqid=client.4566.0:426, at_version=52'40, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.392577+0000", 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.35882949200000003, 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018507790000000001, 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392577+0000", 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392577+0000", 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392578+0000", 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8700000000000005e-07 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393067+0000", 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943399999999996 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393101+0000", 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3951000000000001e-05 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393109+0000", 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5449999999999998e-06 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393182+0000", 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2989000000000002e-05 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393193+0000", 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1167e-05 2024-09-17T17:30:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.394398+0000", 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012048250000000001 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.394428+0000", 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0080999999999999e-05 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 52/35 ECSubWrite(tid=71, reqid=client.4566.0:430, at_version=52'20, trim_to=0'0, roll_forward_to=52'18))", 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.393554+0000", 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.357853011, 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001474387, 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393554+0000", 2024-09-17T17:30:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393554+0000", 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393554+0000", 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4000000000000001e-07 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393572+0000", 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7518000000000001e-05 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393606+0000", 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3785999999999998e-05 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393614+0000", 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0499999999999992e-06 2024-09-17T17:30:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393680+0000", 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5914000000000006e-05 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.393687+0000", 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0079999999999996e-06 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.394998+0000", 2024-09-17T17:30:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001311499 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.395028+0000", 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9872000000000001e-05 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 52/32 ECSubWrite(tid=110, reqid=client.4566.0:436, at_version=52'44, trim_to=0'0, roll_forward_to=52'42))", 2024-09-17T17:30:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.401124+0000", 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.35028282999999999, 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025445490000000001, 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401124+0000", 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401124+0000", 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401124+0000", 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1099999999999996e-07 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401884+0000", 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00076019200000000003 2024-09-17T17:30:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401938+0000", 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3868e-05 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401952+0000", 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3199e-05 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402077+0000", 2024-09-17T17:30:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012546399999999999 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402086+0000", 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3579999999999992e-06 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.403629+0000", 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0015429459999999999 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.403668+0000", 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9011000000000003e-05 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:03.853 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:03.854 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:03.855 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:03.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:03.856 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:03.853+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:03.857 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:03.857 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:03.858 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:03.863 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:03.955 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:03.953+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:04.004 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.004 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:04.056 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:04.054+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:04.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:04.067+0000 7f9233fff640 0 -- 172.21.15.7:0/3719232767 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f923c221650 msgr2=0x7f923c223a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:04.157 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:04.155+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:04.161 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.161 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:04.258 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:04.256+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:04.318 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.318 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:170: done (3 left) 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:171: read oid 43 snap -1 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:171: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:172: read oid 31 snap 8 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:172: expect (ObjNum 61 snap 2 seq_num 61) 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:173: read oid 3 snap -1 2024-09-17T17:30:04.360 INFO:tasks.rados.rados.0.smithi102.stdout:173: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-17T17:30:04.361 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 28 current snap is 10 2024-09-17T17:30:04.361 INFO:tasks.rados.rados.0.smithi102.stdout:168: finishing write tid 1 to smithi10238019-8 2024-09-17T17:30:04.361 INFO:tasks.rados.rados.0.smithi102.stdout:168: finishing write tid 2 to smithi10238019-8 2024-09-17T17:30:04.361 INFO:tasks.rados.rados.0.smithi102.stdout:174: seq_num 90 ranges {1490944=753664,2244608=720896,2965504=49152} 2024-09-17T17:30:04.361 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:04.357+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:04.373 INFO:tasks.rados.rados.0.smithi102.stdout:174: writing smithi10238019-28 from 1490944 to 2244608 tid 1 2024-09-17T17:30:04.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T17:30:04.377 INFO:tasks.rados.rados.0.smithi102.stdout:174: writing smithi10238019-28 from 2244608 to 2965504 tid 2 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:174: writing smithi10238019-28 from 2965504 to 3014656 tid 3 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: finishing write tid 3 to smithi10238019-8 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: finishing write tid 4 to smithi10238019-8 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: finishing write tid 5 to smithi10238019-8 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: oid 8 updating version 0 to 61 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: oid 8 version 61 is already newer than 59 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: oid 8 updating version 61 to 62 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: oid 8 version 62 is already newer than 60 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 8 v 62 (ObjNum 89 snap 10 seq_num 89) dirty exists 2024-09-17T17:30:04.379 INFO:tasks.rados.rados.0.smithi102.stdout:168: left oid 8 (ObjNum 89 snap 10 seq_num 89) 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: finishing write tid 3 to smithi10238019-27 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: finishing write tid 4 to smithi10238019-27 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: finishing write tid 5 to smithi10238019-27 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: oid 27 updating version 0 to 37 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: oid 27 version 37 is already newer than 34 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: oid 27 version 37 is already newer than 36 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: oid 27 version 37 is already newer than 35 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 27 v 37 (ObjNum 87 snap 10 seq_num 87) dirty exists 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:166: left oid 27 (ObjNum 87 snap 10 seq_num 87) 2024-09-17T17:30:04.380 INFO:tasks.rados.rados.0.smithi102.stdout:169: finishing copy_from racing read to smithi10238019-33 2024-09-17T17:30:04.381 INFO:tasks.rados.rados.0.smithi102.stdout:169: finishing copy_from to smithi10238019-33 2024-09-17T17:30:04.381 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 33 v 2 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-17T17:30:04.381 INFO:tasks.rados.rados.0.smithi102.stdout:173: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-17T17:30:04.381 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.381 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.382 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.382 INFO:tasks.rados.rados.0.smithi102.stdout:171: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-17T17:30:04.382 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 58387 2024-09-17T17:30:04.384 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 271442 2024-09-17T17:30:04.385 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.391 INFO:tasks.rados.rados.0.smithi102.stdout:172: expect (ObjNum 61 snap 2 seq_num 61) 2024-09-17T17:30:04.392 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.402 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.411 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:04.422 INFO:tasks.rados.rados.0.smithi102.stdout:166: done (6 left) 2024-09-17T17:30:04.422 INFO:tasks.rados.rados.0.smithi102.stdout:168: done (5 left) 2024-09-17T17:30:04.422 INFO:tasks.rados.rados.0.smithi102.stdout:169: done (4 left) 2024-09-17T17:30:04.422 INFO:tasks.rados.rados.0.smithi102.stdout:171: done (3 left) 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:172: done (2 left) 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:173: done (1 left) 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:175: snap_create 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: finishing write tid 1 to smithi10238019-28 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: finishing write tid 2 to smithi10238019-28 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: finishing write tid 3 to smithi10238019-28 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: finishing write tid 4 to smithi10238019-28 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: finishing write tid 5 to smithi10238019-28 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: oid 28 updating version 0 to 71 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: oid 28 version 71 is already newer than 70 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: oid 28 updating version 71 to 72 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: oid 28 updating version 72 to 73 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 28 v 73 (ObjNum 90 snap 10 seq_num 90) dirty exists 2024-09-17T17:30:04.423 INFO:tasks.rados.rados.0.smithi102.stdout:174: left oid 28 (ObjNum 90 snap 10 seq_num 90) 2024-09-17T17:30:04.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:30:04.459 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.459 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.460 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:04.457+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:04.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:04.492 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:04.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:04.513 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:04.567 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:04.565+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:04.618 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.618 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.627 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:04.661 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:04.659+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:04.785 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:04.783+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:04.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:04.864 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:04.862+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:04.964 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:04.963+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:05.004 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.005 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:05.066 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:05.064+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:05.120 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.120 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:05.167 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:05.165+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:05.235 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.235 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:05.268 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:05.266+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T17:30:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:30:05.332 INFO:tasks.rados.rados.0.smithi102.stdout:174: done (1 left) 2024-09-17T17:30:05.332 INFO:tasks.rados.rados.0.smithi102.stdout:175: done (0 left) 2024-09-17T17:30:05.332 INFO:tasks.rados.rados.0.smithi102.stdout:176: snap_remove snap 10 2024-09-17T17:30:05.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T17:30:05.352 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.352 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:05.388 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:05.386+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:05.466 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.466 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.470 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:05.468+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:05.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:05.571 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:05.568+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:05.583 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.584 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:05.672 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:05.670+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:05.739 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.739 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:05.792 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:05.790+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:05.855 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.856 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.864 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:05.908 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:05.906+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:05.974 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:05.973+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:05.985 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:05.986 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:05.995 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:06.079 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:06.078+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:06.142 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.142 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.150 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:06.176 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:06.174+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:06.278 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:06.276+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:06.294 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.294 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.303 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:06.334 INFO:tasks.rados.rados.0.smithi102.stdout:176: done (0 left) 2024-09-17T17:30:06.334 INFO:tasks.rados.rados.0.smithi102.stdout:177: delete oid 31 current snap is 11 2024-09-17T17:30:06.336 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T17:30:06.339 INFO:tasks.rados.rados.0.smithi102.stdout:177: done (0 left) 2024-09-17T17:30:06.340 INFO:tasks.rados.rados.0.smithi102.stdout:178: read oid 18 snap -1 2024-09-17T17:30:06.340 INFO:tasks.rados.rados.0.smithi102.stdout:178: expect (ObjNum 83 snap 10 seq_num 83) 2024-09-17T17:30:06.340 INFO:tasks.rados.rados.0.smithi102.stdout:179: read oid 13 snap -1 2024-09-17T17:30:06.340 INFO:tasks.rados.rados.0.smithi102.stdout:179: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:06.340 INFO:tasks.rados.rados.0.smithi102.stdout:180: delete oid 5 current snap is 11 2024-09-17T17:30:06.344 INFO:tasks.rados.rados.0.smithi102.stdout:180: done (2 left) 2024-09-17T17:30:06.345 INFO:tasks.rados.rados.0.smithi102.stdout:181: snap_remove snap 9 2024-09-17T17:30:06.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-17T17:30:06.379 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:06.377+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:06.412 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.412 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.421 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:06.480 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:06.478+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:06.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:06.563 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.563 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.571 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:06.581 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:06.579+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:06.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:06.662 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:06.673 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.673 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.681 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:06.682 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:06.681+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:06.706 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:06.752 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:06.785 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.785 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.785 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:06.781+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:06.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:06.817 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:06.837 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:06.859 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:06.883 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:06.888 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:06.886+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:06.896 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:06.897 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:06.904 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:06.905 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:06.924 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:06.925 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:06.947 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:06.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:06.945+0000 7f52b3fff640 0 -- 172.21.15.7:0/3226642280 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f52bc21cf70 msgr2=0x7f52bc21f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-17T17:30:06.990 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:06.989+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:07.051 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=24, last_complete=36'10, committed=1, applied=1))", 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.594122521999999, 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53983539199999997, 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454674+0000", 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0790000000000002e-06 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992637+0000", 2024-09-17T17:30:07.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53796236600000003 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992644+0000", 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5820000000000004e-06 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994469+0000", 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018243350000000001 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994505+0000", 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6300999999999997e-05 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.056 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:173 3.cs0 3:30756a83:::smithi10238019-35:2 [read 297244~276196,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.058699888, 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011614354, 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.057 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 173 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:07.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990093+0000", 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990099+0000", 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2090000000000002e-06 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990111+0000", 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1372000000000001e-05 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990118+0000", 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.126e-06 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990197+0000", 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8665999999999995e-05 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990299+0000", 2024-09-17T17:30:07.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000102397 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001707+0000", 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011407690999999999 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:174 3.cs0 3:30756a83:::smithi10238019-35:2 [read 501714~71726,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.058654429000001, 2024-09-17T17:30:07.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011875784, 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 174 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:07.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9400000000000005e-07 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990142+0000", 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.557e-06 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990195+0000", 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3012e-05 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990206+0000", 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0876000000000001e-05 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990783+0000", 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00057770200000000003 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990850+0000", 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6168999999999996e-05 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.002014+0000", 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011163874000000001 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:175 3.cs0 3:30756a83:::smithi10238019-35:2 [read 194830~378610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:07.067 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.058567871000001, 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.30022850299999998, 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 175 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.068 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990225+0000", 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8199999999999998e-07 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990230+0000", 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7809999999999996e-06 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990371+0000", 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014071299999999999 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990381+0000", 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0061000000000001e-05 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994702+0000", 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043214899999999999 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994769+0000", 2024-09-17T17:30:07.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6600999999999995e-05 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290453+0000", 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.29568397499999999 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=27, attrs_read=0))", 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.992543+0000", 2024-09-17T17:30:07.072 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.056249446999999, 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0088412019999999994, 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992543+0000", 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.073 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992543+0000", 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992543+0000", 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6600000000000002e-07 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992570+0000", 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6707e-05 2024-09-17T17:30:07.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001297+0000", 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0087266759999999992 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001302+0000", 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0379999999999999e-06 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001349+0000", 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7258000000000001e-05 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001384+0000", 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5057e-05 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=28, attrs_read=0))", 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.047594855, 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.289267987, 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.077 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001198+0000", 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6000000000000004e-07 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001279+0000", 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1168000000000006e-05 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289843+0000", 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.28856357599999999 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289854+0000", 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1749999999999999e-05 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289975+0000", 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012060100000000001 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290465+0000", 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049033200000000001 2024-09-17T17:30:07.080 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:194 3.cs0 3:3116c96c:::smithi10238019-11:head [read 168153~12071,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687327+0000", 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 16.361464990000002, 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010349745, 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.081 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 194 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687327+0000", 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687327+0000", 2024-09-17T17:30:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687328+0000", 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2920000000000001e-06 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687335+0000", 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2299999999999996e-06 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687354+0000", 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9794000000000001e-05 2024-09-17T17:30:07.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687364+0000", 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0240000000000003e-06 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.688583+0000", 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0012198770000000001 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.688590+0000", 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.9099999999999999e-06 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691312+0000", 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0027218149999999998 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691398+0000", 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5385999999999999e-05 2024-09-17T17:30:07.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.697677+0000", 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0062794169999999998 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:207 3.cs0 3:30d62bba:::smithi10238019-34:head [copy-from ver 2] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 16.342555789999999, 2024-09-17T17:30:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011310328, 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 207 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706238+0000", 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9230000000000001e-06 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706245+0000", 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5089999999999999e-06 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706261+0000", 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6487999999999999e-05 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706268+0000", 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2859999999999996e-06 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706339+0000", 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1600999999999994e-05 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706451+0000", 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000112071 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.714618+0000", 2024-09-17T17:30:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0081665550000000007 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.715772+0000", 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011545959999999999 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.717464+0000", 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001691641 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.717547+0000", 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2657999999999994e-05 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:208 3.cs0 3:30d62bba:::smithi10238019-34:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.706291+0000", 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 16.342501454000001, 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0092787879999999996, 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.093 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 208 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706291+0000", 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706291+0000", 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706291+0000", 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706296+0000", 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1370000000000001e-06 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706305+0000", 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7649999999999999e-06 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706312+0000", 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4099999999999996e-06 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706644+0000", 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00033282899999999998 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706685+0000", 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0942000000000002e-05 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.715379+0000", 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086938209999999991 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.715473+0000", 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3529999999999994e-05 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.715570+0000", 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6770999999999999e-05 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.098 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 43/32 ECSubWrite(tid=35, reqid=osd.3.0:35, at_version=43'18, trim_to=0'0, roll_forward_to=42'16))", 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.697373+0000", 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.351419318, 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0084875609999999994, 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.099 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697373+0000", 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697373+0000", 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697373+0000", 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1199999999999999e-07 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697392+0000", 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9222e-05 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697436+0000", 2024-09-17T17:30:07.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3825000000000001e-05 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697444+0000", 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0520000000000002e-06 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697519+0000", 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4443000000000005e-05 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697529+0000", 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0655e-05 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.705825+0000", 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00829534 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.705860+0000", 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5712e-05 2024-09-17T17:30:07.103 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.1s1 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.350903538000001, 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011087913, 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.55e-07 2024-09-17T17:30:07.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697911+0000", 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1744999999999999e-05 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697975+0000", 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4414999999999999e-05 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698108+0000", 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013266499999999999 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698174+0000", 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6011000000000002e-05 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698183+0000", 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2539999999999998e-06 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.708938+0000", 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010755602 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.708977+0000", 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8065999999999998e-05 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:313 3.fs0 3:f88f0a4d:::smithi10238019-16:head [append 0~507904] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.981459+0000", 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.067333329, 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010592781000000001, 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 313 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.109 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981459+0000", 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981459+0000", 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981460+0000", 2024-09-17T17:30:07.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.392e-06 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981597+0000", 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013656399999999999 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981610+0000", 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3375e-05 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981618+0000", 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.7600000000000002e-06 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981678+0000", 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9667999999999999e-05 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.981751+0000", 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3658000000000005e-05 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.982364+0000", 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000613018 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.983730+0000", 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001365914 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991975+0000", 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0082448420000000005 2024-09-17T17:30:07.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.992052+0000", 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6589999999999997e-05 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:314 3.fs0 3:f88f0a4d:::smithi10238019-16:head [append 0~507904] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.984391+0000", 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.064401412, 2024-09-17T17:30:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0093131780000000001, 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 314 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984391+0000", 2024-09-17T17:30:07.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984391+0000", 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984392+0000", 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3960000000000001e-06 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984657+0000", 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00026468999999999999 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984667+0000", 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0576e-05 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984673+0000", 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6679999999999998e-06 2024-09-17T17:30:07.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984730+0000", 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6625000000000001e-05 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.984845+0000", 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00011542199999999999 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.985654+0000", 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00080898500000000002 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.986493+0000", 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00083876700000000003 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.993653+0000", 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071602920000000004 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.993704+0000", 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0757000000000002e-05 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:316 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (56) in=7b] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.05933409, 2024-09-17T17:30:07.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041035350999999998, 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 316 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:07.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989459+0000", 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2200000000000004e-07 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989462+0000", 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.287e-06 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989469+0000", 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7229999999999998e-06 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.123 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989472+0000", 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.428e-06 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990430+0000", 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00095784099999999997 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990525+0000", 2024-09-17T17:30:07.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5129000000000003e-05 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990951+0000", 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042552399999999999 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991309+0000", 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000358632 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030441+0000", 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039131753999999998 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030493+0000", 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2510999999999999e-05 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.126 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:317 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.059300754000001, 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041694146000000001, 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 317 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.127 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989492+0000", 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3099999999999998e-07 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989494+0000", 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3329999999999999e-06 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989501+0000", 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.516e-06 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989505+0000", 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.912e-06 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991167+0000", 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001662188 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991214+0000", 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6944000000000003e-05 2024-09-17T17:30:07.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030733+0000", 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039519406999999999 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030782+0000", 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9338000000000001e-05 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.031186+0000", 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000403077 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:405 3.fs0 3:f88f0a4d:::smithi10238019-16:head [rollback 9] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.337960+0000", 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 4.7108325930000001, 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096004760000000001, 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.132 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 405 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.337960+0000", 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.337960+0000", 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.337962+0000", 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1229999999999998e-06 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.337971+0000", 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8060000000000001e-06 2024-09-17T17:30:07.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.337988+0000", 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7399000000000001e-05 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338003+0000", 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4901e-05 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:30:07.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5487000000000003e-05 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338096+0000", 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.841e-06 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340586+0000", 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024895400000000002 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340708+0000", 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012230300000000001 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341643+0000", 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00093491899999999998 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347002+0000", 2024-09-17T17:30:07.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053588050000000003 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347504+0000", 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00050185099999999999 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347560+0000", 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6501e-05 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:406 3.fs0 3:f88f0a4d:::smithi10238019-16:head [append 0~0] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338036+0000", 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 4.7107563140000002, 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010514643000000001, 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 406 2024-09-17T17:30:07.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338036+0000", 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338036+0000", 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338037+0000", 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0999999999999997e-07 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338041+0000", 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3769999999999998e-06 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338051+0000", 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.024e-05 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338058+0000", 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0439999999999998e-06 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338164+0000", 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010580499999999999 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338170+0000", 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1909999999999997e-06 2024-09-17T17:30:07.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342237+0000", 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0040662149999999998 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342348+0000", 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111253 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342887+0000", 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00053960499999999997 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347003+0000", 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041159860000000003 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348476+0000", 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001472983 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348551+0000", 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4134000000000005e-05 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:429 3.cs0 3:30d62bba:::smithi10238019-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:07.145 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 3.655889224, 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014137011, 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 429 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.146 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3299999999999998e-07 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392905+0000", 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6869999999999999e-06 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392912+0000", 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6250000000000001e-06 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392916+0000", 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7330000000000003e-06 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397110+0000", 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041939450000000001 2024-09-17T17:30:07.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397209+0000", 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8961000000000003e-05 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402765+0000", 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055556420000000004 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402840+0000", 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4943000000000004e-05 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.407040+0000", 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042001419999999996 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:436 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 12] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 3.6557923260000003, 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012854290000000001, 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 436 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3700000000000001e-07 2024-09-17T17:30:07.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393005+0000", 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4190000000000004e-06 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393008+0000", 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7009999999999998e-06 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393023+0000", 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4545999999999999e-05 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397903+0000", 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048805719999999997 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397964+0000", 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0520000000000003e-05 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.401036+0000", 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030720470000000001 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402722+0000", 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001685513 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405761+0000", 2024-09-17T17:30:07.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030398790000000001 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405854+0000", 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2756000000000004e-05 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:437 3.cs0 3:349443fd:::smithi10238019-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393031+0000", 2024-09-17T17:30:07.157 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 3.6557612779999999, 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0093873399999999992, 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 437 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:07.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393031+0000", 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393031+0000", 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393031+0000", 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7400000000000003e-07 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393036+0000", 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0409999999999999e-06 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393040+0000", 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.8960000000000002e-06 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393044+0000", 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2260000000000001e-06 2024-09-17T17:30:07.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.398075+0000", 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0050310620000000002 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.398122+0000", 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7419000000000001e-05 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402176+0000", 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0040535700000000003 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402303+0000", 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000127427 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402418+0000", 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00011512500000000001 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.162 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.163 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.163 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:07.163 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:07.163 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:07.164 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:07.165 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:07.162+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:07.265 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:07.262+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:07.310 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:07.310 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:07.318 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.8 to 1 (10000) 2024-09-17T17:30:07.335 INFO:tasks.rados.rados.0.smithi102.stdout:181: done (2 left) 2024-09-17T17:30:07.335 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 29 current snap is 11 2024-09-17T17:30:07.336 INFO:tasks.rados.rados.0.smithi102.stdout:182: seq_num 93 ranges {212992=802816,1015808=638976,1654784=131072} 2024-09-17T17:30:07.342 INFO:tasks.rados.rados.0.smithi102.stdout:182: writing smithi10238019-29 from 212992 to 1015808 tid 1 2024-09-17T17:30:07.346 INFO:tasks.rados.rados.0.smithi102.stdout:182: writing smithi10238019-29 from 1015808 to 1654784 tid 2 2024-09-17T17:30:07.348 INFO:tasks.rados.rados.0.smithi102.stdout:182: writing smithi10238019-29 from 1654784 to 1785856 tid 3 2024-09-17T17:30:07.348 INFO:tasks.rados.rados.0.smithi102.stdout:183: delete oid 28 current snap is 11 2024-09-17T17:30:07.348 INFO:tasks.rados.rados.0.smithi102.stdout:179: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:07.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:30:07.349 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 45246 2024-09-17T17:30:07.352 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 211158 2024-09-17T17:30:07.352 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:07.355 INFO:tasks.rados.rados.0.smithi102.stdout:179: done (3 left) 2024-09-17T17:30:07.355 INFO:tasks.rados.rados.0.smithi102.stdout:183: done (2 left) 2024-09-17T17:30:07.355 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 16 current snap is 11 2024-09-17T17:30:07.355 INFO:tasks.rados.rados.0.smithi102.stdout:184: seq_num 95 ranges {2686976=442368,3129344=540672,3670016=622592} 2024-09-17T17:30:07.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-17T17:30:07.365 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:07.363+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:07.372 INFO:tasks.rados.rados.0.smithi102.stdout:184: writing smithi10238019-16 from 2686976 to 3129344 tid 1 2024-09-17T17:30:07.375 INFO:tasks.rados.rados.0.smithi102.stdout:184: writing smithi10238019-16 from 3129344 to 3670016 tid 2 2024-09-17T17:30:07.379 INFO:tasks.rados.rados.0.smithi102.stdout:184: writing smithi10238019-16 from 3670016 to 4292608 tid 3 2024-09-17T17:30:07.379 INFO:tasks.rados.rados.0.smithi102.stdout:185: snap_remove snap 8 2024-09-17T17:30:07.431 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:07.431 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:07.439 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:07.481 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:07.479+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:07.566 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:07.565+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:07.582 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:07.582 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:07.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:07.668 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:07.666+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:07.697 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:07.697 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:07.697 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:07.698 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:07.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:07.769 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:07.767+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:07.812 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:07.813 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:07.813 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:07.813 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:07.813 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:07.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:07.870 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:07.868+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:07.930 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.243918763, 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020160569999999999, 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0499999999999999e-07 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683889+0000", 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000276748 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683926+0000", 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7307e-05 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683937+0000", 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0445000000000001e-05 2024-09-17T17:30:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684015+0000", 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8745999999999994e-05 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684028+0000", 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2733e-05 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015570569999999999 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2616e-05 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.235646441, 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018516470000000001, 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691884+0000", 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691889+0000", 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6330000000000004e-06 2024-09-17T17:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692145+0000", 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025631599999999998 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692190+0000", 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4969000000000003e-05 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692199+0000", 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4599999999999992e-06 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692277+0000", 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7676999999999995e-05 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692289+0000", 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T17:30:07.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693695+0000", 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014064679999999999 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.693736+0000", 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-05 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 39/32 ECSubWrite(tid=15, reqid=client.4566.0:199, at_version=39'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.227314604, 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001846723, 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:07.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700216+0000", 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700460+0000", 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024369999999999999 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700559+0000", 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8998000000000003e-05 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700569+0000", 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9510000000000001e-06 2024-09-17T17:30:07.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700641+0000", 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2325999999999998e-05 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700656+0000", 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702016+0000", 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013601799999999999 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702063+0000", 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6384999999999997e-05 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=80, reqid=client.4566.0:274, at_version=42'13, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.219612141000001, 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003864363, 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707919+0000", 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708479+0000", 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055997299999999996 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708513+0000", 2024-09-17T17:30:07.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4359000000000001e-05 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708524+0000", 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0346999999999999e-05 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708600+0000", 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999997e-05 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708619+0000", 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8490000000000001e-05 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711752+0000", 2024-09-17T17:30:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031327600000000001 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711783+0000", 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1035999999999999e-05 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=81, reqid=client.4566.0:275, at_version=42'14, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.217244336, 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025720560000000001, 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710287+0000", 2024-09-17T17:30:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3300000000000001e-07 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710492+0000", 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020486500000000001 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710535+0000", 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3510999999999997e-05 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710545+0000", 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9869999999999994e-06 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710609+0000", 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3379000000000004e-05 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710619+0000", 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0098000000000001e-05 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712823+0000", 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022037350000000001 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712858+0000", 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5648000000000002e-05 2024-09-17T17:30:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 42/35 ECSubWrite(tid=86, reqid=client.4566.0:282, at_version=42'30, trim_to=0'0, roll_forward_to=41'28))", 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.193833567, 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002415295, 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733698+0000", 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2759999999999999e-06 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734149+0000", 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045118399999999998 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734186+0000", 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7109e-05 2024-09-17T17:30:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734195+0000", 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0539999999999995e-06 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734270+0000", 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5165000000000003e-05 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734282+0000", 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2763e-05 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736069+0000", 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017863530000000001 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736112+0000", 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3390999999999998e-05 2024-09-17T17:30:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.234798931, 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020892049999999998, 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692761+0000", 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9167999999999999e-05 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692813+0000", 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2231999999999997e-05 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692825+0000", 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2075999999999999e-05 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692965+0000", 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013971200000000001 2024-09-17T17:30:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692978+0000", 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2738999999999999e-05 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694786+0000", 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001808732 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694821+0000", 2024-09-17T17:30:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4363000000000003e-05 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=37, reqid=osd.8.0:37, at_version=43'27, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.234537259, 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001896823, 2024-09-17T17:30:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692993+0000", 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693010+0000", 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6254999999999999e-05 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693046+0000", 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-05 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693055+0000", 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3480000000000004e-06 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693202+0000", 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000147369 2024-09-17T17:30:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693211+0000", 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2499999999999995e-06 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694870+0000", 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016585759999999999 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694890+0000", 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0203999999999998e-05 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.23371395, 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001933894, 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693817+0000", 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693835+0000", 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7604999999999998e-05 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693865+0000", 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0372999999999999e-05 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693877+0000", 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2143999999999999e-05 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693956+0000", 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9281000000000002e-05 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.693968+0000", 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1100999999999999e-05 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.695713+0000", 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001745833 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.695750+0000", 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7015999999999999e-05 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.229963841, 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010715961, 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697586+0000", 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8556999999999999e-05 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697629+0000", 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2982999999999998e-05 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697637+0000", 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697698+0000", 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0534e-05 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697710+0000", 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1745e-05 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708248+0000", 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010538476999999999 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708283+0000", 2024-09-17T17:30:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4431999999999998e-05 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.226679716, 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074300950000000003, 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700852+0000", 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1969999999999999e-06 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700867+0000", 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4960000000000001e-05 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700917+0000", 2024-09-17T17:30:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0285999999999999e-05 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700931+0000", 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3332999999999999e-05 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701036+0000", 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105467 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701054+0000", 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7827999999999999e-05 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708242+0000", 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071881929999999998 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708281+0000", 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8831000000000001e-05 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.226096175, 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068976910000000001, 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701435+0000", 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701447+0000", 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2424e-05 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1851e-05 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701527+0000", 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7069999999999992e-06 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701592+0000", 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4825000000000004e-05 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701600+0000", 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-06 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708305+0000", 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067049680000000004 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708332+0000", 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6899000000000001e-05 2024-09-17T17:30:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 48/32 ECSubWrite(tid=41, reqid=osd.8.0:40, at_version=0'0, trim_to=0'0, roll_forward_to=48'31))", 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.2073065270000001, 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77214092400000001, 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720235+0000", 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0353999999999999e-05 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491642+0000", 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77140694700000001 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491651+0000", 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491754+0000", 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010309 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491768+0000", 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144999999999999e-05 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492327+0000", 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000558998 2024-09-17T17:30:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492365+0000", 2024-09-17T17:30:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969000000000002e-05 2024-09-17T17:30:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 51/35 ECSubWrite(tid=60, reqid=osd.12.0:60, at_version=51'16, trim_to=0'0, roll_forward_to=50'14))", 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5915239349999997, 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028553210000000001, 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336043+0000", 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6211000000000003e-05 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337026+0000", 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098293199999999995 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337033+0000", 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4919999999999996e-06 2024-09-17T17:30:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337104+0000", 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0616000000000003e-05 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337116+0000", 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1454000000000001e-05 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338825+0000", 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001709824 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338862+0000", 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6477000000000003e-05 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 51/32 ECSubWrite(tid=34, reqid=osd.7.0:34, at_version=51'25, trim_to=0'0, roll_forward_to=49'23))", 2024-09-17T17:30:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.336152+0000", 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5913784370000004, 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019889980000000001, 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336152+0000", 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336152+0000", 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336152+0000", 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.17e-07 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336174+0000", 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1622999999999999e-05 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336205+0000", 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1421999999999998e-05 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336215+0000", 2024-09-17T17:30:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6169999999999996e-06 2024-09-17T17:30:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336303+0000", 2024-09-17T17:30:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7744000000000006e-05 2024-09-17T17:30:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336312+0000", 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6479999999999998e-06 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338104+0000", 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001791579 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338141+0000", 2024-09-17T17:30:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7147999999999997e-05 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/32 ECSubWrite(tid=43, reqid=osd.7.0:43, at_version=52'34, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.348698+0000", 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5788321600000002, 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020771689999999998, 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348698+0000", 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348698+0000", 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348699+0000", 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7399999999999991e-07 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348724+0000", 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4247999999999999e-05 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348765+0000", 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1598999999999999e-05 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348782+0000", 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6847e-05 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348908+0000", 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012611100000000001 2024-09-17T17:30:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348940+0000", 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1642e-05 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.350702+0000", 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017624240000000001 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.350775+0000", 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3324000000000005e-05 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 52/35 ECSubWrite(tid=67, reqid=osd.12.0:67, at_version=52'18, trim_to=0'0, roll_forward_to=51'16))", 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.578045908, 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002010759, 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349520+0000", 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5457999999999998e-05 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349577+0000", 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6814000000000002e-05 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349588+0000", 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0495e-05 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349794+0000", 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000206059 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349803+0000", 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8230000000000006e-06 2024-09-17T17:30:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.351465+0000", 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016613999999999999 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.351495+0000", 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0428000000000001e-05 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=49, reqid=osd.7.0:49, at_version=56'40, trim_to=0'0, roll_forward_to=53'38))", 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.59323749800000003, 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022420579999999999, 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1100000000000001e-07 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334316+0000", 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2364000000000001e-05 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334355+0000", 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9201e-05 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334394+0000", 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9391999999999998e-05 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334471+0000", 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6897999999999995e-05 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334482+0000", 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0720000000000001e-05 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.336496+0000", 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020137599999999999 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.336535+0000", 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9311999999999999e-05 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 56/32 ECSubWrite(tid=48, reqid=osd.8.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=56'35))", 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.337849+0000", 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.58968136699999996, 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001943731, 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.337849+0000", 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.337849+0000", 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.337850+0000", 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9999999999999996e-07 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.337863+0000", 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2709e-05 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.337924+0000", 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1531000000000004e-05 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.337932+0000", 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8860000000000004e-06 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338100+0000", 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016738199999999999 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1968e-05 2024-09-17T17:30:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339353+0000", 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012418399999999999 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339793+0000", 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043951500000000002 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.339173+0000", 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.58835754399999995, 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021330870000000001, 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339173+0000", 2024-09-17T17:30:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339173+0000", 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339174+0000", 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4799999999999997e-07 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339193+0000", 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8927e-05 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339230+0000", 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7274000000000003e-05 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339238+0000", 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6249999999999998e-06 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339455+0000", 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000217011 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339463+0000", 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6300000000000004e-06 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018003839999999999 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341306+0000", 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2388000000000001e-05 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.033 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:08.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:08.035 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:08.033+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:08.083 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:08.105 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:08.159 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:08.157+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:08.185 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:08.186 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:08.186 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:08.187 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:08.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:08.236 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:08.235+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:08.337 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:08.335+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:08.342 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:08.342 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:08.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:08.364 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:30:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.9 to 1 (10000) 2024-09-17T17:30:08.370 INFO:tasks.rados.rados.0.smithi102.stdout:185: done (3 left) 2024-09-17T17:30:08.370 INFO:tasks.rados.rados.0.smithi102.stdout:186: rollback oid 41 current snap is 11 2024-09-17T17:30:08.370 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 41 to 7 2024-09-17T17:30:08.370 INFO:tasks.rados.rados.0.smithi102.stdout:178: expect (ObjNum 83 snap 10 seq_num 83) 2024-09-17T17:30:08.370 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 54403 2024-09-17T17:30:08.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-17T17:30:08.397 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 597178 2024-09-17T17:30:08.407 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: finishing write tid 1 to smithi10238019-29 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: finishing write tid 2 to smithi10238019-29 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: finishing write tid 3 to smithi10238019-29 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: finishing write tid 4 to smithi10238019-29 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: finishing write tid 5 to smithi10238019-29 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: oid 29 updating version 0 to 37 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: oid 29 version 37 is already newer than 35 2024-09-17T17:30:08.425 INFO:tasks.rados.rados.0.smithi102.stdout:182: oid 29 version 37 is already newer than 36 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:182: oid 29 updating version 37 to 38 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 29 v 38 (ObjNum 93 snap 11 seq_num 93) dirty exists 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:182: left oid 29 (ObjNum 93 snap 11 seq_num 93) 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: finishing write tid 1 to smithi10238019-16 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: finishing write tid 2 to smithi10238019-16 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: finishing write tid 3 to smithi10238019-16 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:178: done (3 left) 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:182: done (2 left) 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:187: delete oid 24 current snap is 11 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: finishing write tid 4 to smithi10238019-16 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: finishing write tid 5 to smithi10238019-16 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: oid 16 updating version 0 to 46 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: oid 16 updating version 46 to 47 2024-09-17T17:30:08.426 INFO:tasks.rados.rados.0.smithi102.stdout:184: oid 16 version 47 is already newer than 45 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:184: oid 16 updating version 47 to 48 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 48 (ObjNum 95 snap 11 seq_num 95) dirty exists 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:184: left oid 16 (ObjNum 95 snap 11 seq_num 95) 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:186: finishing rollback tid 0 to smithi10238019-41 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:186: finishing rollback tid 1 to smithi10238019-41 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:186: finishing rollback tid 2 to smithi10238019-41 2024-09-17T17:30:08.427 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 41 v 23 (ObjNum 69 snap 4 seq_num 69) dirty exists 2024-09-17T17:30:08.477 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:08.475+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.798659616999998, 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002820571, 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684890+0000", 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3499000000000001e-05 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685111+0000", 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022111 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685121+0000", 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7070000000000004e-06 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685199+0000", 2024-09-17T17:30:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8145999999999993e-05 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685209+0000", 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0138e-05 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687657+0000", 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024481260000000001 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687696+0000", 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.909e-05 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.4s3 39/35 ECSubWrite(tid=11, reqid=osd.1.0:10, at_version=0'0, trim_to=0'0, roll_forward_to=39'8))", 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.798086957999999, 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002246264, 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685448+0000", 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685449+0000", 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0800000000000004e-07 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685461+0000", 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1664999999999999e-05 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685533+0000", 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2149000000000003e-05 2024-09-17T17:30:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685542+0000", 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685619+0000", 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7186000000000003e-05 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:30:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687653+0000", 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020252030000000002 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687694+0000", 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1072000000000002e-05 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.791572156000001, 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022546670000000001, 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8700000000000002e-07 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692554+0000", 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000590721 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692590+0000", 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6350000000000003e-05 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692602+0000", 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692708+0000", 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106835 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692718+0000", 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694181+0000", 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014632149999999999 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694218+0000", 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6730999999999998e-05 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 50/35 ECSubWrite(tid=89, reqid=client.4566.0:401, at_version=50'26, trim_to=0'0, roll_forward_to=49'24))", 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1298536290000003, 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002465409, 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353683+0000", 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2890000000000001e-06 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353717+0000", 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4183000000000001e-05 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353760+0000", 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.316e-05 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353773+0000", 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3037000000000001e-05 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353852+0000", 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8621000000000005e-05 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353862+0000", 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0614e-05 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356113+0000", 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022509050000000001 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356147+0000", 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999997e-05 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=94, reqid=client.4566.0:406, at_version=51'30, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1403242760000003, 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003277556, 2024-09-17T17:30:08.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343212+0000", 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343225+0000", 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414999999999999e-05 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343249+0000", 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3983999999999999e-05 2024-09-17T17:30:08.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343254+0000", 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5249999999999996e-06 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343307+0000", 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2167999999999999e-05 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343315+0000", 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8580000000000006e-06 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346453+0000", 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003137908 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346488+0000", 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5021999999999998e-05 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 51/35 ECSubWrite(tid=62, reqid=osd.12.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.13898206, 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025408480000000001, 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:08.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344554+0000", 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8100000000000001e-07 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344567+0000", 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.254e-05 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344609+0000", 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2549000000000002e-05 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344619+0000", 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6299999999999993e-06 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344683+0000", 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4090999999999993e-05 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344693+0000", 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0363000000000001e-05 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347044+0000", 2024-09-17T17:30:08.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023509339999999998 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347094+0000", 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.986e-05 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=97, reqid=osd.11.0:97, at_version=51'34, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1384893949999997, 2024-09-17T17:30:08.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024083809999999998, 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345065+0000", 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8292e-05 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345091+0000", 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6812999999999999e-05 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345100+0000", 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7010000000000002e-06 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345388+0000", 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028787499999999999 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345396+0000", 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2269999999999993e-06 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347422+0000", 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020255170000000001 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347454+0000", 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2443000000000001e-05 2024-09-17T17:30:08.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.0940176179999996, 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024942340000000001, 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389963+0000", 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044482000000000001 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389993+0000", 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.995e-05 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389999+0000", 2024-09-17T17:30:08.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7830000000000004e-06 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390051+0000", 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2475999999999997e-05 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390057+0000", 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7540000000000003e-06 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.391979+0000", 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001921835 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.392012+0000", 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2848000000000001e-05 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 56/35 ECSubWrite(tid=57, reqid=osd.10.0:57, at_version=56'41, trim_to=0'0, roll_forward_to=53'39))", 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.1482717010000001, 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036979209999999998, 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335264+0000", 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335298+0000", 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3837000000000001e-05 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335654+0000", 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035630699999999999 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335665+0000", 2024-09-17T17:30:08.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1620999999999999e-05 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335751+0000", 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5692999999999994e-05 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335763+0000", 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140000000000001e-05 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338926+0000", 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031629399999999999 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338961+0000", 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5129999999999997e-05 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.1454659869999999, 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003185701, 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338094+0000", 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4428000000000001e-05 2024-09-17T17:30:08.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011644700000000001 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338219+0000", 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9020000000000005e-06 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338416+0000", 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196558 2024-09-17T17:30:08.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338424+0000", 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.585e-06 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341209+0000", 2024-09-17T17:30:08.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027847050000000002 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341255+0000", 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:08.533 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.145424757, 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004124801, 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9600000000000001e-07 2024-09-17T17:30:08.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338129+0000", 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8003e-05 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338243+0000", 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114367 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338249+0000", 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3160000000000002e-06 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338788+0000", 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053876399999999995 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338797+0000", 2024-09-17T17:30:08.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-06 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342196+0000", 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033993830000000002 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342235+0000", 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8921999999999998e-05 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.1453258230000001, 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040538249999999996, 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:08.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.99e-07 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338233+0000", 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3623999999999999e-05 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338275+0000", 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2249000000000002e-05 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338284+0000", 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2609999999999993e-06 2024-09-17T17:30:08.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338436+0000", 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152032 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338448+0000", 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2469999999999999e-05 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342230+0000", 2024-09-17T17:30:08.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037816899999999999 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342263+0000", 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3300000000000003e-05 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.145265212, 2024-09-17T17:30:08.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029941270000000001, 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:08.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338289+0000", 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8340000000000001e-05 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338322+0000", 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3419e-05 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338332+0000", 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0086e-05 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338873+0000", 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054120000000000004 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338882+0000", 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1329999999999999e-06 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341233+0000", 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023510699999999998 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:08.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1365999999999998e-05 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 57/35 ECSubWrite(tid=140, reqid=osd.6.0:140, at_version=57'15, trim_to=0'0, roll_forward_to=56'13))", 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.11700735499999999, 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024870859999999999, 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T17:30:08.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366550+0000", 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1702e-05 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366615+0000", 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5030999999999995e-05 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366623+0000", 2024-09-17T17:30:08.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366683+0000", 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9488999999999999e-05 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366692+0000", 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5089999999999999e-06 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.368974+0000", 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022823029999999998 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.369015+0000", 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0383000000000003e-05 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.11116038, 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285746, 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372399+0000", 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4416000000000001e-05 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372479+0000", 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9556999999999997e-05 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372487+0000", 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7589999999999997e-06 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372561+0000", 2024-09-17T17:30:08.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3807000000000007e-05 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372571+0000", 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0532e-05 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375622+0000", 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030507860000000002 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375661+0000", 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8634000000000003e-05 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.110430614, 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004179131, 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:08.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373125+0000", 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9626000000000001e-05 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373159+0000", 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4190999999999998e-05 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373168+0000", 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6999999999999997e-06 2024-09-17T17:30:08.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373237+0000", 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9306000000000004e-05 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373246+0000", 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377245+0000", 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039987390000000003 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377284+0000", 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8423999999999997e-05 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:08.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.110084496, 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048101309999999996, 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1800000000000005e-07 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373476+0000", 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4712999999999999e-05 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373535+0000", 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8536999999999999e-05 2024-09-17T17:30:08.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373545+0000", 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0142e-05 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374163+0000", 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061771899999999997 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374172+0000", 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.081e-06 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378233+0000", 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040611509999999998 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378261+0000", 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000002e-05 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.110034411, 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038195389999999998, 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373518+0000", 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373595+0000", 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6206999999999997e-05 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373601+0000", 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6039999999999998e-06 2024-09-17T17:30:08.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373659+0000", 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7442000000000003e-05 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373668+0000", 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2040000000000003e-06 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377300+0000", 2024-09-17T17:30:08.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00363182 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377320+0000", 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0598000000000002e-05 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.10743288600000001, 2024-09-17T17:30:08.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034371530000000001, 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376103+0000", 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376128+0000", 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5109e-05 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376199+0000", 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1783e-05 2024-09-17T17:30:08.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376209+0000", 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1710000000000008e-06 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376272+0000", 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749000000000002e-05 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376281+0000", 2024-09-17T17:30:08.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379498+0000", 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003216605 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379539+0000", 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1587e-05 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.377735+0000", 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.105800265, 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024431040000000002, 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377735+0000", 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377735+0000", 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377736+0000", 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377756+0000", 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0415e-05 2024-09-17T17:30:08.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377792+0000", 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6139000000000001e-05 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377802+0000", 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6250000000000002e-06 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378477+0000", 2024-09-17T17:30:08.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067517399999999996 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378486+0000", 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5030000000000003e-06 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380145+0000", 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016589599999999999 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380178+0000", 2024-09-17T17:30:08.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2526e-05 2024-09-17T17:30:08.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.579 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:08.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:08.581 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:08.578+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:184: done (2 left) 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:186: done (1 left) 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:187: done (0 left) 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:188: rollback oid 44 current snap is 11 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 44 to 7 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:189: read oid 11 snap -1 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:189: expect deleted 2024-09-17T17:30:08.659 INFO:tasks.rados.rados.0.smithi102.stdout:190: snap_create 2024-09-17T17:30:08.662 INFO:tasks.rados.rados.0.smithi102.stdout:188: finishing rollback tid 0 to smithi10238019-44 2024-09-17T17:30:08.663 INFO:tasks.rados.rados.0.smithi102.stdout:188: finishing rollback tid 1 to smithi10238019-44 2024-09-17T17:30:08.681 INFO:tasks.rados.rados.0.smithi102.stdout:188: finishing rollback tid 2 to smithi10238019-44 2024-09-17T17:30:08.681 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 44 v 27 (ObjNum 55 snap 2 seq_num 55) dirty exists 2024-09-17T17:30:08.681 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:08.679+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:08.730 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:08.730 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:08.730 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:08.730 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:08.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:08.782 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:08.781+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:08.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:08.899 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:08.897+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:08.964 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.276926192000001, 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:30:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=20, reqid=client.4566.0:191, at_version=39'17, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.271391858000001, 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001787521, 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690192+0000", 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690618+0000", 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042645600000000002 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690656+0000", 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8214999999999999e-05 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690666+0000", 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5419999999999994e-06 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690744+0000", 2024-09-17T17:30:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8418999999999996e-05 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690754+0000", 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9329999999999996e-06 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691939+0000", 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001184198 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691978+0000", 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9845999999999998e-05 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.263390022999999, 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:30:08.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.261164164, 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:30:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.260740084999998, 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:08.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:30:08.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:30:08.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.991 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.260645783000001, 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:30:08.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.260020820000001, 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015435340000000001, 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:08.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701563+0000", 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.155e-06 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701569+0000", 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701604+0000", 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5148999999999998e-05 2024-09-17T17:30:08.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701612+0000", 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9559999999999997e-06 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703084+0000", 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014716270000000001 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703105+0000", 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1766000000000001e-05 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.259783532, 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001340387, 2024-09-17T17:30:09.001 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:09.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701800+0000", 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2900000000000002e-07 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701806+0000", 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5099999999999998e-06 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701957+0000", 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000151175 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701965+0000", 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703127+0000", 2024-09-17T17:30:09.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011621019999999999 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703140+0000", 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.295e-05 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.7s1 39/32 ECSubWrite(tid=57, reqid=client.4566.0:182, at_version=39'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:09.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.258150559000001, 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00185337, 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703433+0000", 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4300000000000002e-07 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704005+0000", 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057213000000000001 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704034+0000", 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8662000000000001e-05 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704039+0000", 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1170000000000003e-06 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704101+0000", 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1818999999999999e-05 2024-09-17T17:30:09.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704111+0000", 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0115000000000001e-05 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705248+0000", 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011371420000000001 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705286+0000", 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7542000000000001e-05 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=35, reqid=osd.3.0:35, at_version=43'18, trim_to=0'0, roll_forward_to=42'16))", 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.264195022999999, 2024-09-17T17:30:09.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001758098, 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:09.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697411+0000", 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2937000000000001e-05 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697449+0000", 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519000000000002e-05 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697458+0000", 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697532+0000", 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4147999999999995e-05 2024-09-17T17:30:09.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697542+0000", 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.893e-06 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699107+0000", 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015656050000000001 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699146+0000", 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8525000000000001e-05 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.bs4 43/32 ECSubWrite(tid=92, reqid=osd.13.0:92, at_version=43'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.263706092, 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019485469999999999, 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697917+0000", 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9153999999999998e-05 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697948+0000", 2024-09-17T17:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1538000000000002e-05 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697957+0000", 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698771+0000", 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081437600000000001 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698781+0000", 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9590000000000007e-06 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699776+0000", 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099482900000000003 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699825+0000", 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9280000000000003e-05 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=37, reqid=client.4566.0:303, at_version=43'20, trim_to=0'0, roll_forward_to=43'18))", 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.994889521999999, 2024-09-17T17:30:09.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013508630000000001, 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966695+0000", 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8390000000000001e-06 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966972+0000", 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027734400000000001 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967017+0000", 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4558000000000003e-05 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967027+0000", 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0431999999999999e-05 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967083+0000", 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5686999999999997e-05 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967090+0000", 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.861e-06 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.968020+0000", 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093037199999999995 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.968044+0000", 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3770000000000001e-05 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=54, reqid=client.4566.0:390, at_version=50'32, trim_to=0'0, roll_forward_to=48'30))", 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.6253604900000003, 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001333426, 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:09.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336223+0000", 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336344+0000", 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121086 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336381+0000", 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6823000000000003e-05 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336391+0000", 2024-09-17T17:30:09.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0725e-05 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336445+0000", 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3505999999999999e-05 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336452+0000", 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4749999999999996e-06 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.337528+0000", 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010753760000000001 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.337556+0000", 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7937999999999999e-05 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=55, reqid=client.4566.0:394, at_version=50'34, trim_to=0'0, roll_forward_to=48'30))", 2024-09-17T17:30:09.030 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.6209676130000004, 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014379869999999999, 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340617+0000", 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340887+0000", 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027011199999999998 2024-09-17T17:30:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340937+0000", 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0489999999999999e-05 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340949+0000", 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2054e-05 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.341016+0000", 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7136000000000002e-05 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.341026+0000", 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.342022+0000", 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099594500000000003 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.342053+0000", 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1619999999999999e-05 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=56, reqid=client.4566.0:395, at_version=50'35, trim_to=0'0, roll_forward_to=50'32))", 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.6174711530000003, 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013727380000000001, 2024-09-17T17:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344113+0000", 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344333+0000", 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022061300000000001 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344418+0000", 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4672000000000005e-05 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344426+0000", 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.79e-06 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344496+0000", 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0178000000000005e-05 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344506+0000", 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0299000000000001e-05 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.345448+0000", 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094149200000000004 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.345484+0000", 2024-09-17T17:30:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6652000000000001e-05 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 51/32 ECSubWrite(tid=61, reqid=osd.3.0:61, at_version=51'39, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.6213916370000003, 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00196635, 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340192+0000", 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340213+0000", 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.145e-05 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340244+0000", 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1063999999999999e-05 2024-09-17T17:30:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340255+0000", 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1066e-05 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8850000000000007e-05 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340334+0000", 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0081999999999999e-05 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.342124+0000", 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001789646 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.342157+0000", 2024-09-17T17:30:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3788999999999998e-05 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 52/32 ECSubWrite(tid=66, reqid=osd.3.0:66, at_version=52'45, trim_to=0'0, roll_forward_to=51'43))", 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.357940+0000", 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6036427140000002, 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001359168, 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357940+0000", 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357940+0000", 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357941+0000", 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2999999999999999e-07 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357969+0000", 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7698000000000001e-05 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358012+0000", 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3310999999999999e-05 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358022+0000", 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0434999999999999e-05 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358126+0000", 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103191 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358137+0000", 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0907000000000001e-05 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.359257+0000", 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001120977 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.359299+0000", 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1718999999999998e-05 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 56/32 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=56'59, trim_to=0'0, roll_forward_to=53'55))", 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.347958+0000", 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.613625189, 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013959630000000001, 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347958+0000", 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347958+0000", 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347959+0000", 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5799999999999999e-06 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347988+0000", 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9329999999999999e-05 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348037+0000", 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8995000000000001e-05 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348048+0000", 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1012999999999999e-05 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348134+0000", 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-05 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348144+0000", 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0251999999999999e-05 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349315+0000", 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011705629999999999 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349354+0000", 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8810000000000003e-05 2024-09-17T17:30:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=82, reqid=osd.3.0:82, at_version=57'61, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.59030792899999995, 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035036519999999999, 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-07 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371301+0000", 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6461e-05 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372066+0000", 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076444900000000005 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372076+0000", 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372145+0000", 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9586999999999996e-05 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372155+0000", 2024-09-17T17:30:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0054999999999999e-05 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374734+0000", 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790520000000001 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374778+0000", 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4042000000000003e-05 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=57'63, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.58821863200000002, 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026238189999999999, 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373365+0000", 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0640000000000001e-06 2024-09-17T17:30:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373399+0000", 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3757000000000002e-05 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373439+0000", 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0028999999999999e-05 2024-09-17T17:30:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373469+0000", 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0297000000000001e-05 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373543+0000", 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4202999999999994e-05 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373574+0000", 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0895999999999997e-05 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375923+0000", 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023489689999999998 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375988+0000", 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4603999999999993e-05 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.064 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:09.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:09.066 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:09.063+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:09.166 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:09.164+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:09.214 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:09.214 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:09.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:09.267 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:09.265+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:09.330 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:09.330 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:09.330 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:09.330 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:09.330 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:09.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:09.368 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:09.367+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:09.376 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.10 to 1 (10000) 2024-09-17T17:30:09.379 INFO:tasks.rados.rados.0.smithi102.stdout:188: done (2 left) 2024-09-17T17:30:09.379 INFO:tasks.rados.rados.0.smithi102.stdout:189: done (1 left) 2024-09-17T17:30:09.379 INFO:tasks.rados.rados.0.smithi102.stdout:190: done (0 left) 2024-09-17T17:30:09.379 INFO:tasks.rados.rados.0.smithi102.stdout:191: snap_create 2024-09-17T17:30:09.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-17T17:30:09.450 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:09.450 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:09.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:09.450 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.764440368999999, 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:30:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:30:09.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:211 3.9s0 3:96cbaefd:::smithi10238019-46:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.7369515, 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068972499999999997, 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 211 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0400000000000002e-07 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710516+0000", 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5269999999999999e-06 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710520+0000", 2024-09-17T17:30:09.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4059999999999998e-06 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710524+0000", 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.518e-06 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712807+0000", 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002283322 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713024+0000", 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216629 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713484+0000", 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046035200000000003 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714391+0000", 2024-09-17T17:30:09.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090667600000000003 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717343+0000", 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002952863 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717411+0000", 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7652999999999998e-05 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.733487146, 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:30:09.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.701185503, 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:30:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:30:09.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.699261001, 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:30:09.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:30:09.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:399 3.0s0 3:00c2d3b1:::smithi10238019-29:head [copy-from ver 2] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1042026919999994, 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01110614, 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 399 2024-09-17T17:30:09.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343265+0000", 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343273+0000", 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343291+0000", 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7764000000000001e-05 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343299+0000", 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1629999999999996e-06 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343366+0000", 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6398999999999999e-05 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343443+0000", 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7731000000000005e-05 2024-09-17T17:30:09.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.352245+0000", 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088010680000000004 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353574+0000", 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013293109999999999 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.354245+0000", 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067153099999999995 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.354369+0000", 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012345600000000001 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:400 3.0s0 3:00c2d3b1:::smithi10238019-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:09.481 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1041472809999995, 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010156996, 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 400 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343319+0000", 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343321+0000", 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5689999999999999e-06 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343335+0000", 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3541e-05 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343343+0000", 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2740000000000007e-06 2024-09-17T17:30:09.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343584+0000", 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241179 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343618+0000", 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4147999999999999e-05 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353248+0000", 2024-09-17T17:30:09.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096298000000000009 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353357+0000", 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109281 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353475+0000", 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011772 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:410 3.9s0 3:93d1baed:::smithi10238019-5:head [copy-from ver 8] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1093648070000004, 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017989010999999999, 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 410 2024-09-17T17:30:09.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338102+0000", 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5349999999999999e-06 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338107+0000", 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000003e-06 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338121+0000", 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3743e-05 2024-09-17T17:30:09.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338129+0000", 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4209999999999995e-06 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8885000000000003e-05 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338278+0000", 2024-09-17T17:30:09.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0192999999999995e-05 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.349495+0000", 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011217173 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350828+0000", 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013328700000000001 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356035+0000", 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052068000000000001 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356090+0000", 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4731000000000002e-05 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:411 3.9s0 3:93d1baed:::smithi10238019-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.1093138079999996, 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012330081, 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 411 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-17T17:30:09.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338155+0000", 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0580000000000002e-06 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338164+0000", 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338170+0000", 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2809999999999997e-06 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338459+0000", 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028850300000000001 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338502+0000", 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3068000000000002e-05 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350266+0000", 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011764431000000001 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350370+0000", 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010380200000000001 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350482+0000", 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011173800000000001 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 52/35 ECSubWriteReply(tid=76, last_complete=52'24, committed=1, applied=1))", 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0379131829999997, 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.18030790999999999, 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409553+0000", 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409560+0000", 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589756+0000", 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.180196197 2024-09-17T17:30:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589765+0000", 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589834+0000", 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9301000000000006e-05 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589860+0000", 2024-09-17T17:30:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6058999999999999e-05 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:477 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~802816] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1054003350000001, 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014281265, 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 477 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342067+0000", 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.906e-06 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343566+0000", 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001499521 2024-09-17T17:30:09.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343581+0000", 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4949e-05 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343590+0000", 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1529999999999997e-06 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343749+0000", 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343862+0000", 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011281399999999999 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.345103+0000", 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001240718 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346311+0000", 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012080210000000001 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356249+0000", 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099386809999999996 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356346+0000", 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7062e-05 2024-09-17T17:30:09.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:478 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~638976] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.346105+0000", 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.101360391, 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010395823, 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 478 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346105+0000", 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346105+0000", 2024-09-17T17:30:09.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346107+0000", 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.911e-06 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346705+0000", 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059845199999999997 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346723+0000", 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8019000000000001e-05 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346733+0000", 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.781e-06 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346777+0000", 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3535999999999998e-05 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346853+0000", 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6148000000000003e-05 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347517+0000", 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066432999999999998 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351003+0000", 2024-09-17T17:30:09.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034862750000000001 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356429+0000", 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005425933 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356501+0000", 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1438000000000003e-05 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:479 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~131072] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.099387659, 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017167436000000001, 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 479 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348079+0000", 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.719e-06 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348181+0000", 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101258 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348196+0000", 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5423000000000001e-05 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348204+0000", 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348898+0000", 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069432499999999998 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349042+0000", 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143848 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349752+0000", 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071013599999999999 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351054+0000", 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301618 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365180+0000", 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014125511 2024-09-17T17:30:09.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365245+0000", 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5647000000000004e-05 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:480 3.0s0 3:00c2d3b1:::smithi10238019-29:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.0992452909999999, 2024-09-17T17:30:09.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017175950999999998, 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 480 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348221+0000", 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348225+0000", 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969999999999998e-06 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348235+0000", 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312000000000001e-05 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348241+0000", 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340000000000002e-06 2024-09-17T17:30:09.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350222+0000", 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019816209999999998 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350336+0000", 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113779 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350754+0000", 2024-09-17T17:30:09.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041740400000000002 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351686+0000", 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093241600000000004 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365336+0000", 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013650254000000001 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365396+0000", 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9704000000000003e-05 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:481 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.0992142089999999, 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018899091, 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 481 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348252+0000", 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348255+0000", 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6720000000000002e-06 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348265+0000", 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.132e-06 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348271+0000", 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351236+0000", 2024-09-17T17:30:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029650309999999999 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351333+0000", 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7449999999999997e-05 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365440+0000", 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106966 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365519+0000", 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8703999999999997e-05 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.367150+0000", 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001631354 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:09.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0748922000000001, 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086831600000000005, 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372590+0000", 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6637999999999999e-05 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373245+0000", 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00065489300000000003 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373254+0000", 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2259999999999994e-06 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373321+0000", 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6711999999999994e-05 2024-09-17T17:30:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373330+0000", 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9360000000000005e-06 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381218+0000", 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078879850000000001 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381256+0000", 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8483999999999999e-05 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.374031+0000", 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.073434558, 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088084159999999995, 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374031+0000", 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374031+0000", 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374032+0000", 2024-09-17T17:30:09.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374055+0000", 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2731999999999999e-05 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374105+0000", 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0405000000000003e-05 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374114+0000", 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9539999999999993e-06 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374812+0000", 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069854100000000001 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374821+0000", 2024-09-17T17:30:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5879999999999998e-06 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382816+0000", 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079951710000000006 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382839+0000", 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3023999999999998e-05 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.374058+0000", 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0734072379999999, 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066865479999999996, 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374058+0000", 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374058+0000", 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374058+0000", 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5400000000000002e-07 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374074+0000", 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5573000000000002e-05 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374149+0000", 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4728999999999996e-05 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374155+0000", 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5350000000000001e-06 2024-09-17T17:30:09.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374214+0000", 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8461000000000002e-05 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374224+0000", 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0154e-05 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380715+0000", 2024-09-17T17:30:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064913849999999997 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380745+0000", 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9456999999999999e-05 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.375339+0000", 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0721259089999999, 2024-09-17T17:30:09.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075275389999999998, 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375339+0000", 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375339+0000", 2024-09-17T17:30:09.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375340+0000", 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2399999999999999e-07 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375362+0000", 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1812999999999999e-05 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375397+0000", 2024-09-17T17:30:09.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5079000000000001e-05 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375405+0000", 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7600000000000008e-06 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375538+0000", 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132693 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375549+0000", 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0624e-05 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382849+0000", 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072999470000000002 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382867+0000", 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8399000000000001e-05 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.377887+0000", 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.069578157, 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068341230000000001, 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:09.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377887+0000", 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377887+0000", 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377888+0000", 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4399999999999999e-07 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377924+0000", 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5784999999999997e-05 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377962+0000", 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7939999999999999e-05 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377970+0000", 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4959999999999996e-06 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380976+0000", 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00300527 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380984+0000", 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3180000000000007e-06 2024-09-17T17:30:09.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.384690+0000", 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037059789999999999 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.384721+0000", 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1491000000000001e-05 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:09.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:09.554 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:09.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:09.556 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:09.553+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:09.656 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:09.655+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:09.677 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:09.677 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:09.677 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:09.677 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:09.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:09.758 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:09.756+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:09.795 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:09.795 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:09.795 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:09.795 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:09.795 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:09.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:09.859 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:09.857+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:09.912 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:09.912 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 21.432941185000001, 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:30:09.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.916 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:30:09.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 20.914674947000002, 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:30:09.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.921 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 19.222957424000001, 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.922 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:30:09.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.924 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:30:09.925 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:203 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~557056] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:09.926 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 19.205756513000001, 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0069058180000000002, 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.927 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 203 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703971+0000", 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4849999999999999e-06 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704582+0000", 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00061129700000000001 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704592+0000", 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0094e-05 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704598+0000", 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6019999999999999e-06 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704655+0000", 2024-09-17T17:30:09.930 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7107000000000001e-05 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704712+0000", 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7435000000000002e-05 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.705382+0000", 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067020899999999995 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.931 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706343+0000", 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00096062299999999997 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710818+0000", 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044745519999999997 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710874+0000", 2024-09-17T17:30:09.932 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6413999999999999e-05 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 19.203514142, 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.933 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.934 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:30:09.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.936 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:30:09.937 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.938 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 19.203127239000001, 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.940 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:30:09.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:30:09.942 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.943 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:30:09.944 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 19.203093872, 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.946 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:30:09.948 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 19.198644400999999, 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.952 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:367 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~589824] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.5710335860000004, 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012183095999999999, 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 367 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.955 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.956 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338693+0000", 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.795e-06 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341609+0000", 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029156210000000002 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341623+0000", 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4583000000000001e-05 2024-09-17T17:30:09.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341630+0000", 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2230000000000001e-06 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341691+0000", 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0878000000000002e-05 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341771+0000", 2024-09-17T17:30:09.958 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9308999999999996e-05 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343069+0000", 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012979420000000001 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348577+0000", 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0055080229999999999 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.959 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350781+0000", 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002204742 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350874+0000", 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2979999999999994e-05 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:368 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~573440] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.567393569, 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0088382370000000005, 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 368 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.961 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342332+0000", 2024-09-17T17:30:09.962 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.08e-06 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342959+0000", 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062686600000000001 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343062+0000", 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010252 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.963 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343083+0000", 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1135e-05 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344260+0000", 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011769969999999999 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344389+0000", 2024-09-17T17:30:09.964 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012922399999999999 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345712+0000", 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001323182 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348773+0000", 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030611649999999998 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.965 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351091+0000", 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023171590000000001 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351169+0000", 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8909e-05 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:369 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~196608] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.5662918599999998, 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0079413130000000002, 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 369 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.968 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343434+0000", 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4699999999999999e-06 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343667+0000", 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023281800000000001 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343680+0000", 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2524000000000001e-05 2024-09-17T17:30:09.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343686+0000", 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.19e-06 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346596+0000", 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002909867 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.970 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346803+0000", 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00020759199999999999 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348042+0000", 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001238636 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350047+0000", 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002004774 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351299+0000", 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001252364 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351374+0000", 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5078000000000002e-05 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:370 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.5660241169999995, 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0082819799999999996, 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 370 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.973 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.974 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3799999999999997e-07 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343710+0000", 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2570000000000007e-06 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343715+0000", 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6829999999999996e-06 2024-09-17T17:30:09.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343721+0000", 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349122+0000", 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005400841 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.976 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349229+0000", 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107076 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349735+0000", 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050621599999999998 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351781+0000", 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020453279999999999 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.977 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351908+0000", 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012790499999999999 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351983+0000", 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4154999999999997e-05 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.978 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:371 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.5659970740000002, 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010270708, 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 371 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:09.980 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4799999999999999e-07 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343730+0000", 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4389999999999999e-06 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343735+0000", 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1919999999999998e-06 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343738+0000", 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1360000000000001e-06 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352473+0000", 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0087356429999999995 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352570+0000", 2024-09-17T17:30:09.982 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6261999999999994e-05 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353998+0000", 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001428688 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s1 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.337395+0000", 2024-09-17T17:30:09.983 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.5723293630000001, 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.006904136, 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337395+0000", 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.984 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337395+0000", 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337396+0000", 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1e-07 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337420+0000", 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3828e-05 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337512+0000", 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2149000000000002e-05 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337520+0000", 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.653e-06 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337598+0000", 2024-09-17T17:30:09.986 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7371000000000002e-05 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337609+0000", 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1567e-05 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344184+0000", 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0065750189999999997 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.987 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344300+0000", 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000115239 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.337972+0000", 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.571753041, 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0068924709999999998, 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337972+0000", 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337972+0000", 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337972+0000", 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4999999999999998e-07 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338000+0000", 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7657e-05 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338049+0000", 2024-09-17T17:30:09.990 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9750000000000003e-05 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338062+0000", 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2388000000000001e-05 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338162+0000", 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010019299999999999 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.991 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338173+0000", 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.084e-05 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344824+0000", 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0066511929999999997 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344864+0000", 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0099999999999999e-05 2024-09-17T17:30:09.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.fs6 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.338327+0000", 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.5713978929999999, 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005912909, 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338327+0000", 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338327+0000", 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338328+0000", 2024-09-17T17:30:09.994 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1699999999999997e-07 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338356+0000", 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7949e-05 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338394+0000", 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8211999999999999e-05 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:09.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338403+0000", 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1460000000000002e-06 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338525+0000", 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000121834 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338536+0000", 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1413000000000001e-05 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344192+0000", 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0056552690000000001 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344240+0000", 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8168999999999999e-05 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.997 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.fs6 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 2.5293468309999998, 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75327243099999996, 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380379+0000", 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8100000000000002e-07 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380396+0000", 2024-09-17T17:30:09.999 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7717e-05 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132975+0000", 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75257808000000004 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132981+0000", 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2249999999999997e-06 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.000 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133046+0000", 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4944999999999996e-05 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133053+0000", 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133623+0000", 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057078100000000002 2024-09-17T17:30:10.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133650+0000", 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6996e-05 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.373082+0000", 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.5366428889999999, 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0063917139999999997, 2024-09-17T17:30:10.002 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373082+0000", 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373082+0000", 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373082+0000", 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.61e-07 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373111+0000", 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8527e-05 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373200+0000", 2024-09-17T17:30:10.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9103000000000004e-05 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373209+0000", 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6190000000000006e-06 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373284+0000", 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4534000000000002e-05 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373295+0000", 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0769e-05 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379415+0000", 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0061208419999999996 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379474+0000", 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8158999999999997e-05 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.006 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:495 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.2516607020000001, 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.022462405000000001, 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:10.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 495 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658065+0000", 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2600000000000001e-07 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658072+0000", 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8430000000000004e-06 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658082+0000", 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.927e-06 2024-09-17T17:30:10.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658089+0000", 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8909999999999998e-06 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660406+0000", 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023178890000000001 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:10.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660504+0000", 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7341000000000002e-05 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661132+0000", 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062810899999999996 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661861+0000", 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072939299999999995 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680403+0000", 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018541588000000001 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680526+0000", 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012359799999999999 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/35 ECSubWriteReply(tid=37, last_complete=57'27, committed=1, applied=1))", 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.247893876, 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019253007999999999, 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:10.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661832+0000", 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:10.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661839+0000", 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7999999999999999e-06 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680112+0000", 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018272335000000001 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680126+0000", 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3867e-05 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680275+0000", 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000149741 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.681084+0000", 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00080847500000000001 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:10.016 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:10.017 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:10.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:10.019 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:10.016+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:10.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:10.119 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:10.117+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:10.125 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:10.126 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:10.126 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:10.126 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:10.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:10.220 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:10.218+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:10.255 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:10.278 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:10.281 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:10.281 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:10.281 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:10.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:10.281 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:10.289 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:10.300 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:10.320 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:10.318+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:10.322 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.11 to 1 (10000) 2024-09-17T17:30:10.333 INFO:tasks.rados.rados.0.smithi102.stdout:191: done (0 left) 2024-09-17T17:30:10.333 INFO:tasks.rados.rados.0.smithi102.stdout:192: delete oid 11 current snap is 13 2024-09-17T17:30:10.344 INFO:tasks.rados.rados.0.smithi102.stdout:192: done (0 left) 2024-09-17T17:30:10.344 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 25 current snap is 13 2024-09-17T17:30:10.344 INFO:tasks.rados.rados.0.smithi102.stdout:193: seq_num 98 ranges {409600=491520,901120=589824,1490944=557056,2048000=16384} 2024-09-17T17:30:10.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-17T17:30:10.349 INFO:tasks.rados.rados.0.smithi102.stdout:193: writing smithi10238019-25 from 409600 to 901120 tid 1 2024-09-17T17:30:10.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:10.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:10.353 INFO:tasks.rados.rados.0.smithi102.stdout:193: writing smithi10238019-25 from 901120 to 1490944 tid 2 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:193: writing smithi10238019-25 from 1490944 to 2048000 tid 3 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:193: writing smithi10238019-25 from 2048000 to 2064384 tid 4 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:194: read oid 41 snap 12 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:194: expect (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 8 current snap is 13 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:193: finishing write tid 1 to smithi10238019-25 2024-09-17T17:30:10.356 INFO:tasks.rados.rados.0.smithi102.stdout:195: seq_num 99 ranges {1638400=507904,2146304=606208,2752512=557056} 2024-09-17T17:30:10.368 INFO:tasks.rados.rados.0.smithi102.stdout:195: writing smithi10238019-8 from 1638400 to 2146304 tid 1 2024-09-17T17:30:10.371 INFO:tasks.rados.rados.0.smithi102.stdout:195: writing smithi10238019-8 from 2146304 to 2752512 tid 2 2024-09-17T17:30:10.375 INFO:tasks.rados.rados.0.smithi102.stdout:195: writing smithi10238019-8 from 2752512 to 3309568 tid 3 2024-09-17T17:30:10.375 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 44 current snap is 13 2024-09-17T17:30:10.375 INFO:tasks.rados.rados.0.smithi102.stdout:193: finishing write tid 2 to smithi10238019-25 2024-09-17T17:30:10.375 INFO:tasks.rados.rados.0.smithi102.stdout:196: seq_num 100 ranges {933888=524288,1458176=753664,2211840=409600} 2024-09-17T17:30:10.378 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:10.383 INFO:tasks.rados.rados.0.smithi102.stdout:196: writing smithi10238019-44 from 933888 to 1458176 tid 1 2024-09-17T17:30:10.387 INFO:tasks.rados.rados.0.smithi102.stdout:196: writing smithi10238019-44 from 1458176 to 2211840 tid 2 2024-09-17T17:30:10.389 INFO:tasks.rados.rados.0.smithi102.stdout:196: writing smithi10238019-44 from 2211840 to 2621440 tid 3 2024-09-17T17:30:10.389 INFO:tasks.rados.rados.0.smithi102.stdout:197: snap_remove snap 7 2024-09-17T17:30:10.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:30:10.402 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:10.403 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:10.421 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:10.419+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:10.425 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:10.425 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:10.445 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:10.446 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:10.452 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:10.452 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:10.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:10.467 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:10.467 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:10.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:10.489 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:10.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:10.522 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:10.519+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:10.535 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:10.557 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:10.568 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:10.568 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:10.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:10.578 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:10.577+0000 7f24a961c640 0 -- 172.21.15.7:0/1578700940 >> [v2:172.21.15.102:6818/259571866,v1:172.21.15.102:6819/259571866] conn(0x7f24a4224a60 msgr2=0x7f24a4226e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-17T17:30:10.623 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:10.621+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:10.745 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:10.746 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:10.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:10.746 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:10.743+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:10.845 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:10.844+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:10.893 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:10.893 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:10.901 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:10.946 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:10.944+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:11.009 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.009 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:11.047 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:11.045+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:11.148 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:11.146+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:11.164 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.165 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:11.249 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:11.247+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:11.280 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.280 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:11.350 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:11.348+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.12 to 1 (10000) 2024-09-17T17:30:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:30:11.392 INFO:tasks.rados.rados.0.smithi102.stdout:197: done (4 left) 2024-09-17T17:30:11.392 INFO:tasks.rados.rados.0.smithi102.stdout:198: setattr oid 2 current snap is 13 2024-09-17T17:30:11.393 INFO:tasks.rados.rados.0.smithi102.stdout:193: finishing write tid 3 to smithi10238019-25 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: finishing write tid 4 to smithi10238019-25 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: finishing write tid 5 to smithi10238019-25 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: finishing write tid 6 to smithi10238019-25 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: oid 25 updating version 0 to 59 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: oid 25 updating version 59 to 62 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: oid 25 version 62 is already newer than 61 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: oid 25 updating version 62 to 63 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: oid 25 version 63 is already newer than 60 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 25 v 63 (ObjNum 98 snap 13 seq_num 98) dirty exists 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:193: left oid 25 (ObjNum 98 snap 13 seq_num 98) 2024-09-17T17:30:11.394 INFO:tasks.rados.rados.0.smithi102.stdout:194: expect (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:30:11.399 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 954007 2024-09-17T17:30:11.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-17T17:30:11.419 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.420 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.420 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 662987 2024-09-17T17:30:11.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:11.439 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 918096 2024-09-17T17:30:11.449 INFO:tasks.rados.rados.0.smithi102.stdout:195: finishing write tid 1 to smithi10238019-8 2024-09-17T17:30:11.449 INFO:tasks.rados.rados.0.smithi102.stdout:195: finishing write tid 2 to smithi10238019-8 2024-09-17T17:30:11.449 INFO:tasks.rados.rados.0.smithi102.stdout:195: finishing write tid 3 to smithi10238019-8 2024-09-17T17:30:11.449 INFO:tasks.rados.rados.0.smithi102.stdout:195: finishing write tid 4 to smithi10238019-8 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:195: finishing write tid 5 to smithi10238019-8 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:195: oid 8 updating version 0 to 90 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:195: oid 8 version 90 is already newer than 89 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:195: oid 8 version 90 is already newer than 88 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:195: oid 8 updating version 90 to 91 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 8 v 91 (ObjNum 99 snap 13 seq_num 99) dirty exists 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:195: left oid 8 (ObjNum 99 snap 13 seq_num 99) 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: finishing write tid 1 to smithi10238019-44 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: finishing write tid 2 to smithi10238019-44 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: finishing write tid 3 to smithi10238019-44 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: finishing write tid 4 to smithi10238019-44 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: finishing write tid 5 to smithi10238019-44 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: oid 44 updating version 0 to 32 2024-09-17T17:30:11.450 INFO:tasks.rados.rados.0.smithi102.stdout:196: oid 44 version 32 is already newer than 31 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:196: oid 44 version 32 is already newer than 30 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:196: oid 44 version 32 is already newer than 29 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 44 v 32 (ObjNum 100 snap 13 seq_num 100) dirty exists 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:196: left oid 44 (ObjNum 100 snap 13 seq_num 100) 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:193: done (4 left) 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:194: done (3 left) 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:195: done (2 left) 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:196: done (1 left) 2024-09-17T17:30:11.451 INFO:tasks.rados.rados.0.smithi102.stdout:199: snap_remove snap 12 2024-09-17T17:30:11.452 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:11.450+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:11.537 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.537 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:11.553 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:11.551+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:11.652 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.653 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.654 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:11.651+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:11.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:11.716 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:11.755 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:11.753+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:11.809 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.809 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:11.856 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:11.854+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:11.924 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:11.924 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:11.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:11.956 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:11.955+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:12.039 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.040 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:12.057 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:12.056+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:12.155 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.155 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.158 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:12.156+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:12.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:12.259 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:12.257+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:12.271 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.271 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:12.364 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:12.362+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:12.401 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.13 to 1 (10000) 2024-09-17T17:30:12.404 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 2 v 71 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-17T17:30:12.404 INFO:tasks.rados.rados.0.smithi102.stdout:198: done (1 left) 2024-09-17T17:30:12.404 INFO:tasks.rados.rados.0.smithi102.stdout:199: done (0 left) 2024-09-17T17:30:12.404 INFO:tasks.rados.rados.0.smithi102.stdout:200: snap_remove snap 11 2024-09-17T17:30:12.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-17T17:30:12.434 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.435 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.444 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:12.461 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:12.459+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:12.562 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:12.560+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:12.590 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.590 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.599 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:12.662 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:12.660+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:12.705 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.706 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:12.764 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:12.762+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:12.861 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.861 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.865 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:12.863+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:12.870 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:12.966 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:12.964+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:12.998 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:13.067 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:13.066+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:13.196 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:13.194+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.263 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:13.305 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:13.303+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:13.370 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:13.368+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.14 to 1 (10000) 2024-09-17T17:30:13.406 INFO:tasks.rados.rados.0.smithi102.stdout:200: done (0 left) 2024-09-17T17:30:13.406 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 35 current snap is 13 2024-09-17T17:30:13.406 INFO:tasks.rados.rados.0.smithi102.stdout:201: seq_num 101 ranges {1867776=540672,2408448=507904,2916352=622592,3538944=32768} 2024-09-17T17:30:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.416 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:13.419 INFO:tasks.rados.rados.0.smithi102.stdout:201: writing smithi10238019-35 from 1867776 to 2408448 tid 1 2024-09-17T17:30:13.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-17T17:30:13.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:30:13.422 INFO:tasks.rados.rados.0.smithi102.stdout:201: writing smithi10238019-35 from 2408448 to 2916352 tid 2 2024-09-17T17:30:13.426 INFO:tasks.rados.rados.0.smithi102.stdout:201: writing smithi10238019-35 from 2916352 to 3538944 tid 3 2024-09-17T17:30:13.427 INFO:tasks.rados.rados.0.smithi102.stdout:201: writing smithi10238019-35 from 3538944 to 3571712 tid 4 2024-09-17T17:30:13.427 INFO:tasks.rados.rados.0.smithi102.stdout:202: copy_from oid 3 from oid 7 current snap is 13 2024-09-17T17:30:13.427 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 10 current snap is 13 2024-09-17T17:30:13.427 INFO:tasks.rados.rados.0.smithi102.stdout:203: seq_num 102 ranges {163840=557056,720896=638976,1359872=524288} 2024-09-17T17:30:13.431 INFO:tasks.rados.rados.0.smithi102.stdout:203: writing smithi10238019-10 from 163840 to 720896 tid 1 2024-09-17T17:30:13.435 INFO:tasks.rados.rados.0.smithi102.stdout:203: writing smithi10238019-10 from 720896 to 1359872 tid 2 2024-09-17T17:30:13.439 INFO:tasks.rados.rados.0.smithi102.stdout:203: writing smithi10238019-10 from 1359872 to 1884160 tid 3 2024-09-17T17:30:13.439 INFO:tasks.rados.rados.0.smithi102.stdout:201: finishing write tid 1 to smithi10238019-35 2024-09-17T17:30:13.439 INFO:tasks.rados.rados.0.smithi102.stdout:201: finishing write tid 2 to smithi10238019-35 2024-09-17T17:30:13.439 INFO:tasks.rados.rados.0.smithi102.stdout:204: setattr oid 11 current snap is 13 2024-09-17T17:30:13.439 INFO:tasks.rados.rados.0.smithi102.stdout:201: finishing write tid 3 to smithi10238019-35 2024-09-17T17:30:13.441 INFO:tasks.rados.rados.0.smithi102.stdout:201: finishing write tid 4 to smithi10238019-35 2024-09-17T17:30:13.441 INFO:tasks.rados.rados.0.smithi102.stdout:205: rollback oid 18 current snap is 13 2024-09-17T17:30:13.441 INFO:tasks.rados.rados.0.smithi102.stdout:205: done (4 left) 2024-09-17T17:30:13.441 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 30 current snap is 13 2024-09-17T17:30:13.441 INFO:tasks.rados.rados.0.smithi102.stdout:206: seq_num 103 ranges {491520=573440,1064960=786432,1851392=376832} 2024-09-17T17:30:13.447 INFO:tasks.rados.rados.0.smithi102.stdout:206: writing smithi10238019-30 from 491520 to 1064960 tid 1 2024-09-17T17:30:13.451 INFO:tasks.rados.rados.0.smithi102.stdout:206: writing smithi10238019-30 from 1064960 to 1851392 tid 2 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:206: writing smithi10238019-30 from 1851392 to 2228224 tid 3 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:207: rollback oid 33 current snap is 13 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:207: done (5 left) 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 21 current snap is 13 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:203: finishing write tid 1 to smithi10238019-10 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:201: finishing write tid 5 to smithi10238019-35 2024-09-17T17:30:13.455 INFO:tasks.rados.rados.0.smithi102.stdout:208: seq_num 104 ranges {344064=589824,933888=540672,1474560=622592} 2024-09-17T17:30:13.460 INFO:tasks.rados.rados.0.smithi102.stdout:208: writing smithi10238019-21 from 344064 to 933888 tid 1 2024-09-17T17:30:13.463 INFO:tasks.rados.rados.0.smithi102.stdout:208: writing smithi10238019-21 from 933888 to 1474560 tid 2 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:208: writing smithi10238019-21 from 1474560 to 2097152 tid 3 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 18 current snap is 13 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:203: finishing write tid 2 to smithi10238019-10 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:201: finishing write tid 6 to smithi10238019-35 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:201: oid 35 updating version 0 to 76 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:201: oid 35 version 76 is already newer than 75 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:201: oid 35 version 76 is already newer than 72 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:201: oid 35 updating version 76 to 78 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:201: oid 35 version 78 is already newer than 77 2024-09-17T17:30:13.467 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 35 v 78 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-09-17T17:30:13.468 INFO:tasks.rados.rados.0.smithi102.stdout:201: left oid 35 (ObjNum 101 snap 13 seq_num 101) 2024-09-17T17:30:13.468 INFO:tasks.rados.rados.0.smithi102.stdout:203: finishing write tid 3 to smithi10238019-10 2024-09-17T17:30:13.468 INFO:tasks.rados.rados.0.smithi102.stdout:202: finishing copy_from racing read to smithi10238019-3 2024-09-17T17:30:13.468 INFO:tasks.rados.rados.0.smithi102.stdout:209: seq_num 105 ranges {1703936=606208,2310144=688128,2998272=475136} 2024-09-17T17:30:13.470 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:13.468+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:13.479 INFO:tasks.rados.rados.0.smithi102.stdout:209: writing smithi10238019-18 from 1703936 to 2310144 tid 1 2024-09-17T17:30:13.483 INFO:tasks.rados.rados.0.smithi102.stdout:209: writing smithi10238019-18 from 2310144 to 2998272 tid 2 2024-09-17T17:30:13.487 INFO:tasks.rados.rados.0.smithi102.stdout:209: writing smithi10238019-18 from 2998272 to 3473408 tid 3 2024-09-17T17:30:13.487 INFO:tasks.rados.rados.0.smithi102.stdout:201: done (6 left) 2024-09-17T17:30:13.487 INFO:tasks.rados.rados.0.smithi102.stdout:210: rmattr oid 33 current snap is 13 2024-09-17T17:30:13.487 INFO:tasks.rados.rados.0.smithi102.stdout:210: done (6 left) 2024-09-17T17:30:13.487 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 2 current snap is 13 2024-09-17T17:30:13.487 INFO:tasks.rados.rados.0.smithi102.stdout:211: seq_num 106 ranges {32768=622592,655360=442368,1097728=524288,1622016=196608} 2024-09-17T17:30:13.491 INFO:tasks.rados.rados.0.smithi102.stdout:211: writing smithi10238019-2 from 32768 to 655360 tid 1 2024-09-17T17:30:13.494 INFO:tasks.rados.rados.0.smithi102.stdout:211: writing smithi10238019-2 from 655360 to 1097728 tid 2 2024-09-17T17:30:13.497 INFO:tasks.rados.rados.0.smithi102.stdout:211: writing smithi10238019-2 from 1097728 to 1622016 tid 3 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:211: writing smithi10238019-2 from 1622016 to 1818624 tid 4 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:212: snap_create 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:202: finishing copy_from to smithi10238019-3 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 3 v 28 (ObjNum 76 snap 4 seq_num 76) dirty exists 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:206: finishing write tid 1 to smithi10238019-30 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:206: finishing write tid 2 to smithi10238019-30 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:206: finishing write tid 3 to smithi10238019-30 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:206: finishing write tid 4 to smithi10238019-30 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:206: finishing write tid 5 to smithi10238019-30 2024-09-17T17:30:13.499 INFO:tasks.rados.rados.0.smithi102.stdout:206: oid 30 updating version 0 to 50 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:206: oid 30 version 50 is already newer than 49 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:206: oid 30 version 50 is already newer than 48 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:206: oid 30 version 50 is already newer than 47 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 30 v 50 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:206: left oid 30 (ObjNum 103 snap 13 seq_num 103) 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:208: finishing write tid 1 to smithi10238019-21 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:208: finishing write tid 2 to smithi10238019-21 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:208: finishing write tid 3 to smithi10238019-21 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:209: finishing write tid 1 to smithi10238019-18 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:209: finishing write tid 2 to smithi10238019-18 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:209: finishing write tid 3 to smithi10238019-18 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:209: finishing write tid 4 to smithi10238019-18 2024-09-17T17:30:13.500 INFO:tasks.rados.rados.0.smithi102.stdout:209: finishing write tid 5 to smithi10238019-18 2024-09-17T17:30:13.501 INFO:tasks.rados.rados.0.smithi102.stdout:209: oid 18 updating version 0 to 76 2024-09-17T17:30:13.501 INFO:tasks.rados.rados.0.smithi102.stdout:209: oid 18 updating version 76 to 78 2024-09-17T17:30:13.501 INFO:tasks.rados.rados.0.smithi102.stdout:209: oid 18 version 78 is already newer than 77 2024-09-17T17:30:13.501 INFO:tasks.rados.rados.0.smithi102.stdout:209: oid 18 updating version 78 to 79 2024-09-17T17:30:13.501 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 18 v 79 (ObjNum 105 snap 13 seq_num 105) dirty exists 2024-09-17T17:30:13.501 INFO:tasks.rados.rados.0.smithi102.stdout:209: left oid 18 (ObjNum 105 snap 13 seq_num 105) 2024-09-17T17:30:13.503 INFO:tasks.rados.rados.0.smithi102.stdout:211: finishing write tid 1 to smithi10238019-2 2024-09-17T17:30:13.503 INFO:tasks.rados.rados.0.smithi102.stdout:211: finishing write tid 2 to smithi10238019-2 2024-09-17T17:30:13.504 INFO:tasks.rados.rados.0.smithi102.stdout:211: finishing write tid 3 to smithi10238019-2 2024-09-17T17:30:13.506 INFO:tasks.rados.rados.0.smithi102.stdout:211: finishing write tid 4 to smithi10238019-2 2024-09-17T17:30:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.552 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:13.577 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 11 v 79 (ObjNum 129 snap 0 seq_num 2569168208) dirty exists 2024-09-17T17:30:13.578 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:13.576+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:13.672 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:13.670+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.689 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:13.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:13.773 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:13.771+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:13.776 INFO:tasks.rados.rados.0.smithi102.stdout:208: finishing write tid 4 to smithi10238019-21 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:208: finishing write tid 5 to smithi10238019-21 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:208: oid 21 updating version 0 to 41 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:208: oid 21 version 41 is already newer than 40 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:208: oid 21 updating version 41 to 42 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:208: oid 21 version 42 is already newer than 39 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 21 v 42 (ObjNum 104 snap 13 seq_num 104) dirty exists 2024-09-17T17:30:13.778 INFO:tasks.rados.rados.0.smithi102.stdout:208: left oid 21 (ObjNum 104 snap 13 seq_num 104) 2024-09-17T17:30:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.841 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:13.883 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:13.881+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:13.884 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:13.905 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:13.906 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:13.927 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:13.947 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:13.951 INFO:tasks.rados.rados.0.smithi102.stdout:203: finishing write tid 4 to smithi10238019-10 2024-09-17T17:30:13.952 INFO:tasks.rados.rados.0.smithi102.stdout:203: finishing write tid 5 to smithi10238019-10 2024-09-17T17:30:13.952 INFO:tasks.rados.rados.0.smithi102.stdout:203: oid 10 updating version 0 to 104 2024-09-17T17:30:13.952 INFO:tasks.rados.rados.0.smithi102.stdout:203: oid 10 updating version 104 to 107 2024-09-17T17:30:13.953 INFO:tasks.rados.rados.0.smithi102.stdout:203: oid 10 version 107 is already newer than 105 2024-09-17T17:30:13.953 INFO:tasks.rados.rados.0.smithi102.stdout:203: oid 10 version 107 is already newer than 106 2024-09-17T17:30:13.953 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 10 v 107 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-17T17:30:13.953 INFO:tasks.rados.rados.0.smithi102.stdout:203: left oid 10 (ObjNum 102 snap 13 seq_num 102) 2024-09-17T17:30:13.969 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:13.974 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:13.973+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:13.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:13.993 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:14.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:14.076 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:14.074+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:14.103 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:14.178 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:14.175+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:14.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:14.233 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:14.234 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:14.255 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:14.277 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:14.278 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:14.276+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:14.298 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:14.318 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:14.326 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:14.340 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:14.340 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:14.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:14.344+0000 7f4c8cea9640 0 -- 172.21.15.7:0/1163968097 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f4c882217f0 msgr2=0x7f4c88223be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:14.369 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:14.369 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:14.378 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:14.376+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:14.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.15 to 1 (10000) 2024-09-17T17:30:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:202: done (7 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:203: done (6 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:204: done (5 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:206: done (4 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:208: done (3 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:209: done (2 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:212: done (1 left) 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:213: rollback oid 9 current snap is 14 2024-09-17T17:30:14.417 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 9 to 13 2024-09-17T17:30:14.418 INFO:tasks.rados.rados.0.smithi102.stdout:214: snap_create 2024-09-17T17:30:14.423 INFO:tasks.rados.rados.0.smithi102.stdout:213: finishing rollback tid 0 to smithi10238019-9 2024-09-17T17:30:14.424 INFO:tasks.rados.rados.0.smithi102.stdout:213: finishing rollback tid 1 to smithi10238019-9 2024-09-17T17:30:14.425 INFO:tasks.rados.rados.0.smithi102.stdout:213: finishing rollback tid 2 to smithi10238019-9 2024-09-17T17:30:14.425 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 42 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-17T17:30:14.430 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T17:30:14.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 37/35 ECSubWriteReply(tid=17, last_complete=37'13, committed=1, applied=1))", 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.430396822999999, 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.068701408000000005, 2024-09-17T17:30:14.434 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000937+0000", 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6449999999999998e-06 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068896+0000", 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.067959031000000003 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068909+0000", 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3416e-05 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069008+0000", 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8690999999999994e-05 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.437 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069634+0000", 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062589099999999995 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 23.747465564999999, 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079875469999999994, 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.438 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.439 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2e-07 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684336+0000", 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047134800000000001 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684439+0000", 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010291800000000001 2024-09-17T17:30:14.440 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684449+0000", 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.027e-05 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684596+0000", 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000146749 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.441 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684607+0000", 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1421e-05 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691787+0000", 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071804069999999998 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691851+0000", 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3913999999999998e-05 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.442 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685294+0000", 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 23.746035093, 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0066107179999999998, 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.443 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685294+0000", 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685294+0000", 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685295+0000", 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.065e-06 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.444 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685313+0000", 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8029999999999998e-05 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686226+0000", 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000912775 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686236+0000", 2024-09-17T17:30:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4369999999999996e-06 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686307+0000", 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1019999999999994e-05 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686316+0000", 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1719999999999996e-06 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691864+0000", 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055479930000000002 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691905+0000", 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1226000000000001e-05 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:14.447 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 23.745975159, 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089994210000000005, 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:14.448 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4100000000000001e-07 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685373+0000", 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9063999999999999e-05 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.449 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687383+0000", 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002009355 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687391+0000", 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9699999999999999e-06 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687457+0000", 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5834999999999996e-05 2024-09-17T17:30:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9990000000000003e-06 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694321+0000", 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068546010000000001 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.451 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694354+0000", 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2456000000000002e-05 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=48, reqid=client.4566.0:188, at_version=39'20, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.686729+0000", 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 23.744600483999999, 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057058059999999999, 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.452 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686729+0000", 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686729+0000", 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.453 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686730+0000", 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4700000000000001e-07 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687708+0000", 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00097885200000000011 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.688033+0000", 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00032419700000000003 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.454 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.688042+0000", 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.312e-06 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.688199+0000", 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000157453 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.688209+0000", 2024-09-17T17:30:14.455 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7780000000000002e-06 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.692400+0000", 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041910740000000004 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.692435+0000", 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4393000000000001e-05 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.456 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:191 3.5s0 3:a4d61510:::smithi10238019-32:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 23.743892296999999, 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.008961732, 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:14.457 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 191 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.458 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687439+0000", 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1799999999999999e-06 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687504+0000", 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4770000000000005e-05 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687516+0000", 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2007e-05 2024-09-17T17:30:14.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687531+0000", 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4803e-05 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687595+0000", 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3697999999999999e-05 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:14.460 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687756+0000", 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000161846 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.690070+0000", 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0023137639999999998 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695782+0000", 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057116959999999996 2024-09-17T17:30:14.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695908+0000", 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000126339 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.696399+0000", 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049062900000000002 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:407 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.093252147999999, 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011701008000000001, 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 407 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.463 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.464 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338080+0000", 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2819999999999999e-06 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6289999999999999e-06 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338108+0000", 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9514e-05 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338117+0000", 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.132e-06 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1446999999999995e-05 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338196+0000", 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6350000000000006e-06 2024-09-17T17:30:14.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341655+0000", 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034593860000000001 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342199+0000", 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054402400000000005 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.467 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349778+0000", 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075789589999999997 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:408 3.5s0 3:a96b502c:::smithi10238019-42:head [read 53220~651292,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.093090221000001, 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012064669, 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.468 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 408 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.469 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338240+0000", 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338257+0000", 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6643e-05 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.470 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338270+0000", 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3081000000000001e-05 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338277+0000", 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3780000000000003e-06 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338331+0000", 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4286999999999997e-05 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.471 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338338+0000", 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3609999999999997e-06 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342681+0000", 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043432749999999997 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342775+0000", 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4392e-05 2024-09-17T17:30:14.472 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.350304+0000", 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007528435 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:409 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.092977466000001, 2024-09-17T17:30:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012665852, 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 409 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.474 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.475 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338353+0000", 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.09e-07 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338358+0000", 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7859999999999996e-06 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338366+0000", 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6049999999999998e-06 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338372+0000", 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1739999999999997e-06 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338439+0000", 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7039999999999995e-05 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338446+0000", 2024-09-17T17:30:14.477 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1160000000000001e-06 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343114+0000", 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0046672179999999999 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343207+0000", 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3170000000000004e-05 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.478 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351018+0000", 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078109340000000003 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 51/35 ECSubWrite(tid=122, reqid=osd.13.0:122, at_version=51'53, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.087293658, 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073809449999999999, 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.479 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.480 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1399999999999996e-07 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344057+0000", 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0273000000000001e-05 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344096+0000", 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9045000000000003e-05 2024-09-17T17:30:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344104+0000", 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2050000000000002e-06 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344311+0000", 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00020691500000000001 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344321+0000", 2024-09-17T17:30:14.482 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0307e-05 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351381+0000", 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0070600910000000001 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351417+0000", 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5395000000000002e-05 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.483 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 51/35 ECSubWrite(tid=42, reqid=osd.10.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.348547+0000", 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.082782545000001, 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0058217779999999997, 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348547+0000", 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.484 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348547+0000", 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348548+0000", 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.85e-07 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.485 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348579+0000", 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1615000000000001e-05 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348646+0000", 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6927000000000005e-05 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348655+0000", 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7979999999999995e-06 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.486 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348726+0000", 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1358000000000004e-05 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348736+0000", 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3880000000000006e-06 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.354310+0000", 2024-09-17T17:30:14.487 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055741970000000004 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.354369+0000", 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8610000000000003e-05 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:14.488 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.078823086, 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.988933536, 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.489 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352507+0000", 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1500000000000003e-07 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.490 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352532+0000", 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4485999999999999e-05 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340513+0000", 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.98798163100000003 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340522+0000", 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9379999999999998e-06 2024-09-17T17:30:14.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340625+0000", 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010273 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340633+0000", 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.492 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341398+0000", 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00076522600000000004 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341440+0000", 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1548999999999998e-05 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 51/35 ECSubWrite(tid=66, reqid=client.4566.0:410, at_version=0'0, trim_to=0'0, roll_forward_to=51'31))", 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356289+0000", 2024-09-17T17:30:14.493 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.075040315000001, 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.006488148, 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356289+0000", 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356289+0000", 2024-09-17T17:30:14.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356290+0000", 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2900000000000002e-07 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356299+0000", 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1919999999999994e-06 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.495 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356315+0000", 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5739999999999998e-05 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356321+0000", 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1199999999999999e-06 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356371+0000", 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9697000000000002e-05 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.496 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356378+0000", 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0600000000000002e-06 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.362735+0000", 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063571859999999999 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.362777+0000", 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2324000000000003e-05 2024-09-17T17:30:14.497 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 51/32 ECSubWrite(tid=64, reqid=osd.3.0:64, at_version=51'43, trim_to=0'0, roll_forward_to=51'41))", 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.359380+0000", 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.071949570999999, 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057324719999999997, 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.498 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359380+0000", 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359380+0000", 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359380+0000", 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.89e-07 2024-09-17T17:30:14.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359401+0000", 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0435e-05 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359497+0000", 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6675000000000005e-05 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359504+0000", 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4799999999999998e-06 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359778+0000", 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027401500000000001 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.359786+0000", 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5250000000000005e-06 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.501 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.365074+0000", 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005287829 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.365112+0000", 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8124000000000003e-05 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.502 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/35 ECSubWrite(tid=50, reqid=client.4566.0:441, at_version=0'0, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.567765566, 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.121580543, 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.503 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863565+0000", 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6729999999999999e-06 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863588+0000", 2024-09-17T17:30:14.504 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2756e-05 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984520+0000", 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.120931489 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984547+0000", 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6877999999999999e-05 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.505 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984631+0000", 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4246000000000006e-05 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984638+0000", 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3509999999999996e-06 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985117+0000", 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047851700000000003 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985144+0000", 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7633000000000001e-05 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.0s2 53/35 ECSubWrite(tid=79, reqid=osd.12.0:79, at_version=53'28, trim_to=0'0, roll_forward_to=52'24))", 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.07286781, 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0064458709999999997, 2024-09-17T17:30:14.507 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4199999999999996e-07 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358483+0000", 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0854000000000001e-05 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358510+0000", 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7033999999999999e-05 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358517+0000", 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2099999999999996e-06 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358568+0000", 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0621999999999997e-05 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358576+0000", 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0660000000000004e-06 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.364850+0000", 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0062745359999999998 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.364907+0000", 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7006999999999998e-05 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.511 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 53/32 ECSubWrite(tid=71, reqid=osd.3.0:71, at_version=53'51, trim_to=0'0, roll_forward_to=52'49))", 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.361725+0000", 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.069604532, 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0060243709999999997, 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.512 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361725+0000", 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361725+0000", 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361725+0000", 2024-09-17T17:30:14.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7599999999999999e-07 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361746+0000", 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1270000000000001e-05 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361825+0000", 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8553999999999994e-05 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.514 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361834+0000", 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1409999999999994e-06 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361890+0000", 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6341000000000001e-05 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.361899+0000", 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9979999999999992e-06 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.367715+0000", 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0058156830000000003 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.367749+0000", 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4208000000000002e-05 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.516 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 53/32 ECSubWrite(tid=72, reqid=osd.3.0:72, at_version=53'53, trim_to=0'0, roll_forward_to=52'49))", 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.068781251000001, 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0070743660000000003, 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.517 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362549+0000", 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5400000000000001e-07 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.518 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362568+0000", 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9213000000000001e-05 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362606+0000", 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7647000000000001e-05 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362614+0000", 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2579999999999995e-06 2024-09-17T17:30:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363252+0000", 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00063853300000000005 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363262+0000", 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9359999999999994e-06 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369572+0000", 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063095340000000003 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369622+0000", 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0590999999999997e-05 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 53/35 ECSubWrite(tid=134, reqid=osd.13.0:134, at_version=53'66, trim_to=0'0, roll_forward_to=52'62))", 2024-09-17T17:30:14.521 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.363414+0000", 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.067915352, 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0059597000000000001, 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363414+0000", 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363414+0000", 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363415+0000", 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1799999999999995e-07 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363433+0000", 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8654000000000001e-05 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.523 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363456+0000", 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2412000000000002e-05 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363481+0000", 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4913999999999999e-05 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363577+0000", 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6657999999999995e-05 2024-09-17T17:30:14.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363587+0000", 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3999999999999998e-06 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369334+0000", 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057474400000000004 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369374+0000", 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9604000000000002e-05 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 53/32 ECSubWrite(tid=34, reqid=osd.14.0:34, at_version=53'30, trim_to=0'0, roll_forward_to=52'28))", 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.067895500000001, 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0062252610000000002, 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.526 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:14.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.68e-07 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363456+0000", 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2242000000000002e-05 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363643+0000", 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000186682 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.528 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363652+0000", 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5189999999999998e-06 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363740+0000", 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7960000000000005e-05 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.529 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363753+0000", 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2985e-05 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369627+0000", 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0058739049999999996 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369659+0000", 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.18e-05 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.530 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.531 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.531 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:14.531 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:14.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:14.533 INFO:tasks.rados.rados.0.smithi102.stdout:211: finishing write tid 5 to smithi10238019-2 2024-09-17T17:30:14.533 INFO:tasks.rados.rados.0.smithi102.stdout:211: finishing write tid 6 to smithi10238019-2 2024-09-17T17:30:14.533 INFO:tasks.rados.rados.0.smithi102.stdout:211: oid 2 updating version 0 to 81 2024-09-17T17:30:14.533 INFO:tasks.rados.rados.0.smithi102.stdout:211: oid 2 updating version 81 to 84 2024-09-17T17:30:14.534 INFO:tasks.rados.rados.0.smithi102.stdout:211: oid 2 version 84 is already newer than 83 2024-09-17T17:30:14.534 INFO:tasks.rados.rados.0.smithi102.stdout:211: oid 2 version 84 is already newer than 80 2024-09-17T17:30:14.534 INFO:tasks.rados.rados.0.smithi102.stdout:211: oid 2 version 84 is already newer than 82 2024-09-17T17:30:14.534 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 2 v 84 (ObjNum 106 snap 13 seq_num 106) dirty exists 2024-09-17T17:30:14.534 INFO:tasks.rados.rados.0.smithi102.stdout:211: left oid 2 (ObjNum 106 snap 13 seq_num 106) 2024-09-17T17:30:14.534 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:14.530+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:14.633 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:14.632+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:14.641 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:14.641 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:14.641 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:14.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:14.734 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:14.732+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:14.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:14.835 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:14.834+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:30:14.852 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-17T17:30:14.413096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66208307,"num_objects":52,"num_object_clones":6,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":2,"num_read":367,"num_read_kb":48114,"num_write":591,"num_write_kb":85458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":810,"ondisk_log_size":810,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":558424,"kb_used_data":107160,"kb_used_omap":23,"kb_used_meta":450856,"kb_avail":1415019176,"statfs":{"total":1449551462400,"available":1448979636224,"internally_reserved":0,"allocated":109731840,"data_stored":116869607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":461677194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4767744,"num_objects":-18,"num_object_clones":-17,"num_object_copies":-126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-18,"num_whiteouts":-1,"num_read":11,"num_read_kb":2996,"num_write":79,"num_write_kb":16032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.080673"},"pg_stats":[{"pgid":"3.f","version":"60'56","reported_seq":219,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.421473+0000","last_change":"2024-09-17T17:30:13.421473+0000","last_active":"2024-09-17T17:30:13.421473+0000","last_peered":"2024-09-17T17:30:13.421473+0000","last_clean":"2024-09-17T17:30:13.421473+0000","last_became_active":"2024-09-17T17:29:48.191466+0000","last_became_peered":"2024-09-17T17:29:48.191466+0000","last_unstale":"2024-09-17T17:30:13.421473+0000","last_undegraded":"2024-09-17T17:30:13.421473+0000","last_fullsized":"2024-09-17T17:30:13.421473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.350540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050831599999999998,"stat_sum":{"num_bytes":6979584,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":292,"num_write":40,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,6,13,1],"acting":[11,12,4,14,6,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.e","version":"57'37","reported_seq":170,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.559751+0000","last_change":"2024-09-17T17:30:08.383888+0000","last_active":"2024-09-17T17:30:09.559751+0000","last_peered":"2024-09-17T17:30:09.559751+0000","last_clean":"2024-09-17T17:30:09.559751+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:09.559751+0000","last_undegraded":"2024-09-17T17:30:09.559751+0000","last_fullsized":"2024-09-17T17:30:09.559751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.521377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089048649999999997,"stat_sum":{"num_bytes":5423104,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":16,"num_read_kb":3128,"num_write":21,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"4"},{"start":"a","length":"3"}]},{"pgid":"3.d","version":"60'48","reported_seq":194,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T17:30:12.403325+0000","last_change":"2024-09-17T17:30:12.403733+0000","last_active":"2024-09-17T17:30:12.403325+0000","last_peered":"2024-09-17T17:30:12.403325+0000","last_clean":"2024-09-17T17:30:12.403325+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:30:12.403325+0000","last_undegraded":"2024-09-17T17:30:12.403325+0000","last_fullsized":"2024-09-17T17:30:12.403325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:59.782056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074316900000000001,"stat_sum":{"num_bytes":5062656,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3609,"num_write":37,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"4"},{"start":"e","length":"1"}]},{"pgid":"3.c","version":"62'79","reported_seq":249,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.444056+0000","last_change":"2024-09-17T17:30:13.438667+0000","last_active":"2024-09-17T17:30:13.444056+0000","last_peered":"2024-09-17T17:30:13.444056+0000","last_clean":"2024-09-17T17:30:13.444056+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:13.444056+0000","last_undegraded":"2024-09-17T17:30:13.444056+0000","last_fullsized":"2024-09-17T17:30:13.444056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.589875+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.017718680000000001,"stat_sum":{"num_bytes":7651328,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":3628,"num_write":56,"num_write_kb":10144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.b","version":"60'12","reported_seq":172,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.427709+0000","last_change":"2024-09-17T17:30:13.428111+0000","last_active":"2024-09-17T17:30:13.427709+0000","last_peered":"2024-09-17T17:30:13.427709+0000","last_clean":"2024-09-17T17:30:13.427709+0000","last_became_active":"2024-09-17T17:29:44.653855+0000","last_became_peered":"2024-09-17T17:29:44.653855+0000","last_unstale":"2024-09-17T17:30:13.427709+0000","last_undegraded":"2024-09-17T17:30:13.427709+0000","last_fullsized":"2024-09-17T17:30:13.427709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076427600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":6718,"num_write":6,"num_write_kb":1456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,10,3,8,7,14],"acting":[13,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.a","version":"62'84","reported_seq":247,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.504012+0000","last_change":"2024-09-17T17:30:13.456791+0000","last_active":"2024-09-17T17:30:13.504012+0000","last_peered":"2024-09-17T17:30:13.504012+0000","last_clean":"2024-09-17T17:30:13.504012+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:13.504012+0000","last_undegraded":"2024-09-17T17:30:13.504012+0000","last_fullsized":"2024-09-17T17:30:13.504012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:48.483980+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0061481139999999997,"stat_sum":{"num_bytes":6651904,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":7941,"num_write":70,"num_write_kb":7456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.9","version":"55'36","reported_seq":173,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.377103+0000","last_change":"2024-09-17T17:30:08.366190+0000","last_active":"2024-09-17T17:30:09.377103+0000","last_peered":"2024-09-17T17:30:09.377103+0000","last_clean":"2024-09-17T17:30:09.377103+0000","last_became_active":"2024-09-17T17:29:48.188156+0000","last_became_peered":"2024-09-17T17:29:48.188156+0000","last_unstale":"2024-09-17T17:30:09.377103+0000","last_undegraded":"2024-09-17T17:30:09.377103+0000","last_fullsized":"2024-09-17T17:30:09.377103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:49.996501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068130399999999998,"stat_sum":{"num_bytes":3309568,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":17,"num_write_kb":3888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,6,10,4,15,9,1],"acting":[12,6,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"5","length":"4"},{"start":"a","length":"3"}]},{"pgid":"3.8","version":"60'17","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.404165+0000","last_change":"2024-09-17T17:30:13.404503+0000","last_active":"2024-09-17T17:30:13.404165+0000","last_peered":"2024-09-17T17:30:13.404165+0000","last_clean":"2024-09-17T17:30:13.404165+0000","last_became_active":"2024-09-17T17:29:48.217378+0000","last_became_peered":"2024-09-17T17:29:48.217378+0000","last_unstale":"2024-09-17T17:30:13.404165+0000","last_undegraded":"2024-09-17T17:30:13.404165+0000","last_fullsized":"2024-09-17T17:30:13.404165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:06.788932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000588022,"stat_sum":{"num_bytes":2031616,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":7599,"num_write":8,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,3],"acting":[6,1,0,4,5,15,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"1.0","version":"25'112","reported_seq":222,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.404052+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:30:13.404052+0000","last_peered":"2024-09-17T17:30:13.404052+0000","last_clean":"2024-09-17T17:30:13.404052+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:13.404052+0000","last_undegraded":"2024-09-17T17:30:13.404052+0000","last_fullsized":"2024-09-17T17:30:13.404052+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":88,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432079+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:30:13.432079+0000","last_peered":"2024-09-17T17:30:13.432079+0000","last_clean":"2024-09-17T17:30:13.432079+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:13.432079+0000","last_undegraded":"2024-09-17T17:30:13.432079+0000","last_fullsized":"2024-09-17T17:30:13.432079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"60'38","reported_seq":186,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432366+0000","last_change":"2024-09-17T17:30:13.432686+0000","last_active":"2024-09-17T17:30:13.432366+0000","last_peered":"2024-09-17T17:30:13.432366+0000","last_clean":"2024-09-17T17:30:13.432366+0000","last_became_active":"2024-09-17T17:29:44.654917+0000","last_became_peered":"2024-09-17T17:29:44.654917+0000","last_unstale":"2024-09-17T17:30:13.432366+0000","last_undegraded":"2024-09-17T17:30:13.432366+0000","last_fullsized":"2024-09-17T17:30:13.432366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:55.544812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063548700000000001,"stat_sum":{"num_bytes":2228224,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,10,11],"acting":[14,12,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.450508+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:30:13.450508+0000","last_peered":"2024-09-17T17:30:13.450508+0000","last_clean":"2024-09-17T17:30:13.450508+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:13.450508+0000","last_undegraded":"2024-09-17T17:30:13.450508+0000","last_fullsized":"2024-09-17T17:30:13.450508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"60'25","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.427834+0000","last_change":"2024-09-17T17:30:13.428271+0000","last_active":"2024-09-17T17:30:13.427834+0000","last_peered":"2024-09-17T17:30:13.427834+0000","last_clean":"2024-09-17T17:30:13.427834+0000","last_became_active":"2024-09-17T17:29:48.191774+0000","last_became_peered":"2024-09-17T17:29:48.191774+0000","last_unstale":"2024-09-17T17:30:13.427834+0000","last_undegraded":"2024-09-17T17:30:13.427834+0000","last_fullsized":"2024-09-17T17:30:13.427834+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:45.076435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000921293,"stat_sum":{"num_bytes":2818048,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2994,"num_write":32,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,4],"acting":[13,11,14,10,12,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432115+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:30:13.432115+0000","last_peered":"2024-09-17T17:30:13.432115+0000","last_clean":"2024-09-17T17:30:13.432115+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:13.432115+0000","last_undegraded":"2024-09-17T17:30:13.432115+0000","last_fullsized":"2024-09-17T17:30:13.432115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"57'46","reported_seq":190,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.377098+0000","last_change":"2024-09-17T17:30:08.383767+0000","last_active":"2024-09-17T17:30:09.377098+0000","last_peered":"2024-09-17T17:30:09.377098+0000","last_clean":"2024-09-17T17:30:09.377098+0000","last_became_active":"2024-09-17T17:29:48.191827+0000","last_became_peered":"2024-09-17T17:29:48.191827+0000","last_unstale":"2024-09-17T17:30:09.377098+0000","last_undegraded":"2024-09-17T17:30:09.377098+0000","last_fullsized":"2024-09-17T17:30:09.377098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:04.392403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018298901999999999,"stat_sum":{"num_bytes":3178496,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":18,"num_read_kb":1334,"num_write":17,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,0,6,14,9],"acting":[12,11,15,0,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"5","length":"4"},{"start":"a","length":"3"}]},{"pgid":"2.2","version":"28'2","reported_seq":89,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.450656+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:13.450656+0000","last_peered":"2024-09-17T17:30:13.450656+0000","last_clean":"2024-09-17T17:30:13.450656+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:13.450656+0000","last_undegraded":"2024-09-17T17:30:13.450656+0000","last_fullsized":"2024-09-17T17:30:13.450656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"62'50","reported_seq":226,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.459596+0000","last_change":"2024-09-17T17:30:13.428694+0000","last_active":"2024-09-17T17:30:13.459596+0000","last_peered":"2024-09-17T17:30:13.459596+0000","last_clean":"2024-09-17T17:30:13.459596+0000","last_became_active":"2024-09-17T17:29:48.420194+0000","last_became_peered":"2024-09-17T17:29:48.420194+0000","last_unstale":"2024-09-17T17:30:13.459596+0000","last_undegraded":"2024-09-17T17:30:13.459596+0000","last_fullsized":"2024-09-17T17:30:13.459596+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.669683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086478900000000003,"stat_sum":{"num_bytes":7012352,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":4338,"num_write":25,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,5,3],"acting":[10,7,1,15,4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.5","version":"40'20","reported_seq":197,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.433460+0000","last_change":"2024-09-17T17:30:13.433759+0000","last_active":"2024-09-17T17:30:13.433460+0000","last_peered":"2024-09-17T17:30:13.433460+0000","last_clean":"2024-09-17T17:30:13.433460+0000","last_became_active":"2024-09-17T17:29:48.422075+0000","last_became_peered":"2024-09-17T17:29:48.422075+0000","last_unstale":"2024-09-17T17:30:13.433460+0000","last_undegraded":"2024-09-17T17:30:13.433460+0000","last_fullsized":"2024-09-17T17:30:13.433460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:56.174154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057357800000000004,"stat_sum":{"num_bytes":1736704,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3370,"num_write":32,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,3,7,1,8],"acting":[15,13,5,3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T17:30:11.387801+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:30:11.387801+0000","last_peered":"2024-09-17T17:30:11.387801+0000","last_clean":"2024-09-17T17:30:11.387801+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:30:11.387801+0000","last_undegraded":"2024-09-17T17:30:11.387801+0000","last_fullsized":"2024-09-17T17:30:11.387801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"60'34","reported_seq":187,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T17:30:11.392213+0000","last_change":"2024-09-17T17:30:11.393345+0000","last_active":"2024-09-17T17:30:11.392213+0000","last_peered":"2024-09-17T17:30:11.392213+0000","last_clean":"2024-09-17T17:30:11.392213+0000","last_became_active":"2024-09-17T17:29:48.188164+0000","last_became_peered":"2024-09-17T17:29:48.188164+0000","last_unstale":"2024-09-17T17:30:11.392213+0000","last_undegraded":"2024-09-17T17:30:11.392213+0000","last_fullsized":"2024-09-17T17:30:11.392213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:02.096250+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0055942199999999996,"stat_sum":{"num_bytes":2965504,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2742,"num_write":18,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,4,6,10,13],"acting":[1,12,9,4,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.559638+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:09.559638+0000","last_peered":"2024-09-17T17:30:09.559638+0000","last_clean":"2024-09-17T17:30:09.559638+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:09.559638+0000","last_undegraded":"2024-09-17T17:30:09.559638+0000","last_fullsized":"2024-09-17T17:30:09.559638+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":160,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.428188+0000","last_change":"2024-09-17T17:30:13.428598+0000","last_active":"2024-09-17T17:30:13.428188+0000","last_peered":"2024-09-17T17:30:13.428188+0000","last_clean":"2024-09-17T17:30:13.428188+0000","last_became_active":"2024-09-17T17:29:44.655172+0000","last_became_peered":"2024-09-17T17:29:44.655172+0000","last_unstale":"2024-09-17T17:30:13.428188+0000","last_undegraded":"2024-09-17T17:30:13.428188+0000","last_fullsized":"2024-09-17T17:30:13.428188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:04.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000803902,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":3,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.424097+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:13.424097+0000","last_peered":"2024-09-17T17:30:13.424097+0000","last_clean":"2024-09-17T17:30:13.424097+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:13.424097+0000","last_undegraded":"2024-09-17T17:30:13.424097+0000","last_fullsized":"2024-09-17T17:30:13.424097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"62'107","reported_seq":274,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.443343+0000","last_change":"2024-09-17T17:30:13.439398+0000","last_active":"2024-09-17T17:30:13.443343+0000","last_peered":"2024-09-17T17:30:13.443343+0000","last_clean":"2024-09-17T17:30:13.443343+0000","last_became_active":"2024-09-17T17:29:48.419866+0000","last_became_peered":"2024-09-17T17:29:48.419866+0000","last_unstale":"2024-09-17T17:30:13.443343+0000","last_undegraded":"2024-09-17T17:30:13.443343+0000","last_fullsized":"2024-09-17T17:30:13.443343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:56.022859+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011268261,"stat_sum":{"num_bytes":7979008,"num_objects":6,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":340,"num_write":49,"num_write_kb":15056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,15,1,7,0],"acting":[13,14,6,15,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.433098+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:13.433098+0000","last_peered":"2024-09-17T17:30:13.433098+0000","last_clean":"2024-09-17T17:30:13.433098+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:13.433098+0000","last_undegraded":"2024-09-17T17:30:13.433098+0000","last_fullsized":"2024-09-17T17:30:13.433098+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":65617920,"num_objects":48,"num_object_clones":6,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":2,"num_read":281,"num_read_kb":48041,"num_write":444,"num_write_kb":82784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106274816,"data_stored":114106368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":695,"ondisk_log_size":695,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39504,"kb_used_data":8308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332336,"statfs":{"total":96636764160,"available":96596312064,"internally_reserved":0,"allocated":8507392,"data_stored":11562174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38880,"kb_used_data":7688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332960,"statfs":{"total":96636764160,"available":96596951040,"internally_reserved":0,"allocated":7872512,"data_stored":7601658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247820,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34960,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336880,"statfs":{"total":96636764160,"available":96600965120,"internally_reserved":0,"allocated":3858432,"data_stored":3923134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313228,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45908,"kb_used_data":10556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325932,"statfs":{"total":96636764160,"available":96589754368,"internally_reserved":0,"allocated":10809344,"data_stored":13471048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34040,"kb_used_data":7008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337800,"statfs":{"total":96636764160,"available":96601907200,"internally_reserved":0,"allocated":7176192,"data_stored":7204346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313229,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36588,"kb_used_data":5392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335252,"statfs":{"total":96636764160,"available":96599298048,"internally_reserved":0,"allocated":5521408,"data_stored":5563801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280525,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":6868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":7032832,"data_stored":7057434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36820,"kb_used_data":9784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":10018816,"data_stored":10382842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42988,"kb_used_data":7636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328852,"statfs":{"total":96636764160,"available":96592744448,"internally_reserved":0,"allocated":7819264,"data_stored":7431755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280525,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5693440,"data_stored":5643789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280524,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":5128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":5251072,"data_stored":5111290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,7,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":6536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":6692864,"data_stored":6729210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":94489280524,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":5042176,"data_stored":4664845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39108,"kb_used_data":7912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332732,"statfs":{"total":96636764160,"available":96596717568,"internally_reserved":0,"allocated":8101888,"data_stored":10579134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247820,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41288,"kb_used_data":10092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330552,"statfs":{"total":96636764160,"available":96594485248,"internally_reserved":0,"allocated":10334208,"data_stored":9943147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6090752,"data_stored":6090752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8114176,"data_stored":8859648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6180864,"data_stored":6180864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7327744,"data_stored":10162176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6963200,"data_stored":6963200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":9682944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4698112,"data_stored":5115904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":3084288,"data_stored":3506176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5713920,"data_stored":5713920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6832128,"data_stored":6832128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8347648,"data_stored":8347648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":7733248,"data_stored":11145216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":10473472,"data_stored":10473472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6766592,"data_stored":6766592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8265728,"data_stored":8265728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:30:14.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.231805066, 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020160569999999999, 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683612+0000", 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0499999999999999e-07 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683889+0000", 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000276748 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683926+0000", 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7307e-05 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683937+0000", 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0445000000000001e-05 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684015+0000", 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8745999999999994e-05 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684028+0000", 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2733e-05 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685585+0000", 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015570569999999999 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685628+0000", 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2616e-05 2024-09-17T17:30:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=80, reqid=client.4566.0:274, at_version=42'13, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.207498443999999, 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003864363, 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707919+0000", 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708479+0000", 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055997299999999996 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708513+0000", 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4359000000000001e-05 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708524+0000", 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0346999999999999e-05 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708600+0000", 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999997e-05 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708619+0000", 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8490000000000001e-05 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711752+0000", 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031327600000000001 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711783+0000", 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1035999999999999e-05 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=81, reqid=client.4566.0:275, at_version=42'14, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.205130639, 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025720560000000001, 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710287+0000", 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3300000000000001e-07 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710492+0000", 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020486500000000001 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710535+0000", 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3510999999999997e-05 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710545+0000", 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9869999999999994e-06 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710609+0000", 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3379000000000004e-05 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710619+0000", 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0098000000000001e-05 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712823+0000", 2024-09-17T17:30:14.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022037350000000001 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712858+0000", 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5648000000000002e-05 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 42/35 ECSubWrite(tid=86, reqid=client.4566.0:282, at_version=42'30, trim_to=0'0, roll_forward_to=41'28))", 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:14.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.181719869999998, 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002415295, 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733698+0000", 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2759999999999999e-06 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734149+0000", 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045118399999999998 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734186+0000", 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7109e-05 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734195+0000", 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0539999999999995e-06 2024-09-17T17:30:14.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734270+0000", 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5165000000000003e-05 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734282+0000", 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2763e-05 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736069+0000", 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017863530000000001 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736112+0000", 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3390999999999998e-05 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.222685234, 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020892049999999998, 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692732+0000", 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692761+0000", 2024-09-17T17:30:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9167999999999999e-05 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692813+0000", 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2231999999999997e-05 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692825+0000", 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2075999999999999e-05 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692965+0000", 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013971200000000001 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.692978+0000", 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2738999999999999e-05 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694786+0000", 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001808732 2024-09-17T17:30:14.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.694821+0000", 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4363000000000003e-05 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.217850144, 2024-09-17T17:30:14.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010715961, 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697586+0000", 2024-09-17T17:30:14.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8556999999999999e-05 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697629+0000", 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2982999999999998e-05 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697637+0000", 2024-09-17T17:30:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697698+0000", 2024-09-17T17:30:14.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0534e-05 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697710+0000", 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1745e-05 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708248+0000", 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010538476999999999 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708283+0000", 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4431999999999998e-05 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.948 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.214566018999999, 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074300950000000003, 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700852+0000", 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1969999999999999e-06 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700867+0000", 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4960000000000001e-05 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700917+0000", 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0285999999999999e-05 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700931+0000", 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3332999999999999e-05 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701036+0000", 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105467 2024-09-17T17:30:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701054+0000", 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7827999999999999e-05 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708242+0000", 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071881929999999998 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708281+0000", 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8831000000000001e-05 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.213982477999998, 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068976910000000001, 2024-09-17T17:30:14.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:14.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701435+0000", 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701447+0000", 2024-09-17T17:30:14.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2424e-05 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1851e-05 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701527+0000", 2024-09-17T17:30:14.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7069999999999992e-06 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701592+0000", 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4825000000000004e-05 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701600+0000", 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-06 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708305+0000", 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067049680000000004 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708332+0000", 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6899000000000001e-05 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 48/32 ECSubWrite(tid=41, reqid=osd.8.0:40, at_version=0'0, trim_to=0'0, roll_forward_to=48'31))", 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.19519283, 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77214092400000001, 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-17T17:30:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720235+0000", 2024-09-17T17:30:14.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0353999999999999e-05 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491642+0000", 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77140694700000001 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491651+0000", 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491754+0000", 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010309 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491768+0000", 2024-09-17T17:30:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144999999999999e-05 2024-09-17T17:30:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492327+0000", 2024-09-17T17:30:14.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000558998 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492365+0000", 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969000000000002e-05 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 51/35 ECSubWrite(tid=60, reqid=osd.12.0:60, at_version=51'16, trim_to=0'0, roll_forward_to=50'14))", 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:14.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.579410237999999, 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028553210000000001, 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336043+0000", 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6211000000000003e-05 2024-09-17T17:30:14.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337026+0000", 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098293199999999995 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337033+0000", 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4919999999999996e-06 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337104+0000", 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0616000000000003e-05 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337116+0000", 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1454000000000001e-05 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338825+0000", 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001709824 2024-09-17T17:30:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338862+0000", 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6477000000000003e-05 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/32 ECSubWrite(tid=43, reqid=osd.7.0:43, at_version=52'34, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.348698+0000", 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.566718463000001, 2024-09-17T17:30:14.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020771689999999998, 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348698+0000", 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348698+0000", 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348699+0000", 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7399999999999991e-07 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348724+0000", 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4247999999999999e-05 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348765+0000", 2024-09-17T17:30:14.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1598999999999999e-05 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348782+0000", 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6847e-05 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348908+0000", 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012611100000000001 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.348940+0000", 2024-09-17T17:30:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1642e-05 2024-09-17T17:30:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.350702+0000", 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017624240000000001 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.350775+0000", 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3324000000000005e-05 2024-09-17T17:30:14.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 52/35 ECSubWrite(tid=67, reqid=osd.12.0:67, at_version=52'18, trim_to=0'0, roll_forward_to=51'16))", 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.565932211, 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002010759, 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349485+0000", 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349520+0000", 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5457999999999998e-05 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349577+0000", 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6814000000000002e-05 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349588+0000", 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0495e-05 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349794+0000", 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000206059 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.349803+0000", 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8230000000000006e-06 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.351465+0000", 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016613999999999999 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.351495+0000", 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0428000000000001e-05 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=49, reqid=osd.7.0:49, at_version=56'40, trim_to=0'0, roll_forward_to=53'38))", 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5811238010000004, 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022420579999999999, 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334293+0000", 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1100000000000001e-07 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334316+0000", 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2364000000000001e-05 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334355+0000", 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9201e-05 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334394+0000", 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9391999999999998e-05 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334471+0000", 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6897999999999995e-05 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.334482+0000", 2024-09-17T17:30:14.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0720000000000001e-05 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.336496+0000", 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020137599999999999 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.336535+0000", 2024-09-17T17:30:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9311999999999999e-05 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.339173+0000", 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5762438469999998, 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021330870000000001, 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.992 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339173+0000", 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339173+0000", 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339174+0000", 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4799999999999997e-07 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339193+0000", 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8927e-05 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339230+0000", 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7274000000000003e-05 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339238+0000", 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6249999999999998e-06 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339455+0000", 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000217011 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.339463+0000", 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6300000000000004e-06 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018003839999999999 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341306+0000", 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2388000000000001e-05 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.5403268060000004, 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002547812, 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9299999999999999e-07 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375115+0000", 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.427e-05 2024-09-17T17:30:14.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375207+0000", 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2684999999999998e-05 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375215+0000", 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.824e-06 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:14.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375311+0000", 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6266000000000003e-05 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375323+0000", 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1557e-05 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377599+0000", 2024-09-17T17:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022765390000000002 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377638+0000", 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8377999999999997e-05 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 57/32 ECSubWrite(tid=51, reqid=osd.7.0:51, at_version=57'42, trim_to=0'0, roll_forward_to=56'40))", 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376507+0000", 2024-09-17T17:30:15.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.5389092819999997, 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002326781, 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376507+0000", 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376507+0000", 2024-09-17T17:30:15.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376508+0000", 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7399999999999998e-07 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376533+0000", 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4862999999999999e-05 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376573+0000", 2024-09-17T17:30:15.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0494999999999998e-05 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376585+0000", 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1895e-05 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376671+0000", 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6006000000000003e-05 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376682+0000", 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-05 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378797+0000", 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021148429999999999 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378834+0000", 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7385000000000002e-05 2024-09-17T17:30:15.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 57/32 ECSubWrite(tid=49, reqid=osd.8.0:49, at_version=57'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.5385288800000003, 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026169420000000001, 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2600000000000001e-07 2024-09-17T17:30:15.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376909+0000", 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0905999999999999e-05 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376963+0000", 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.393e-05 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376970+0000", 2024-09-17T17:30:15.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1130000000000003e-06 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377048+0000", 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8391000000000002e-05 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377057+0000", 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379423+0000", 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023655939999999999 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379505+0000", 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2259e-05 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 57/32 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=57'44, trim_to=0'0, roll_forward_to=56'40))", 2024-09-17T17:30:15.010 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.5383679790000002, 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002350857, 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377068+0000", 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8235000000000001e-05 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377097+0000", 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9501000000000001e-05 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377105+0000", 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377158+0000", 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3390000000000002e-05 2024-09-17T17:30:15.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377166+0000", 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2630000000000003e-06 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379364+0000", 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021974960000000002 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379400+0000", 2024-09-17T17:30:15.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5992999999999999e-05 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.5370733879999996, 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028134560000000002, 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378344+0000", 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.47e-07 2024-09-17T17:30:15.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378373+0000", 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.917e-05 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378410+0000", 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6259e-05 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378418+0000", 2024-09-17T17:30:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5299999999999996e-06 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378488+0000", 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9487999999999996e-05 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378497+0000", 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3889999999999994e-06 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381121+0000", 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002624103 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381157+0000", 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.557e-05 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.019 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/35 ECSubWrite(tid=150, reqid=client.4566.0:491, at_version=57'23, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.379854+0000", 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.535563249, 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020009469999999999, 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379854+0000", 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379854+0000", 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379854+0000", 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6500000000000008e-07 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379872+0000", 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7512000000000001e-05 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379904+0000", 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2353e-05 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379919+0000", 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4474000000000001e-05 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379981+0000", 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2069000000000005e-05 2024-09-17T17:30:15.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379991+0000", 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0314e-05 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381825+0000", 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00183358 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381854+0000", 2024-09-17T17:30:15.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.968e-05 2024-09-17T17:30:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.024 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:15.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:15.026 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:15.023+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:15.126 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:15.124+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:15.174 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:15.174 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:15.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:30:15.209 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-17T17:30:14.413096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66208307,"num_objects":52,"num_object_clones":6,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":2,"num_read":367,"num_read_kb":48114,"num_write":591,"num_write_kb":85458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":810,"ondisk_log_size":810,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":558424,"kb_used_data":107160,"kb_used_omap":23,"kb_used_meta":450856,"kb_avail":1415019176,"statfs":{"total":1449551462400,"available":1448979636224,"internally_reserved":0,"allocated":109731840,"data_stored":116869607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":461677194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4767744,"num_objects":-18,"num_object_clones":-17,"num_object_copies":-126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-18,"num_whiteouts":-1,"num_read":11,"num_read_kb":2996,"num_write":79,"num_write_kb":16032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.080673"},"pg_stats":[{"pgid":"3.f","version":"60'56","reported_seq":219,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.421473+0000","last_change":"2024-09-17T17:30:13.421473+0000","last_active":"2024-09-17T17:30:13.421473+0000","last_peered":"2024-09-17T17:30:13.421473+0000","last_clean":"2024-09-17T17:30:13.421473+0000","last_became_active":"2024-09-17T17:29:48.191466+0000","last_became_peered":"2024-09-17T17:29:48.191466+0000","last_unstale":"2024-09-17T17:30:13.421473+0000","last_undegraded":"2024-09-17T17:30:13.421473+0000","last_fullsized":"2024-09-17T17:30:13.421473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.350540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050831599999999998,"stat_sum":{"num_bytes":6979584,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":292,"num_write":40,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,6,13,1],"acting":[11,12,4,14,6,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.e","version":"57'37","reported_seq":170,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.559751+0000","last_change":"2024-09-17T17:30:08.383888+0000","last_active":"2024-09-17T17:30:09.559751+0000","last_peered":"2024-09-17T17:30:09.559751+0000","last_clean":"2024-09-17T17:30:09.559751+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:09.559751+0000","last_undegraded":"2024-09-17T17:30:09.559751+0000","last_fullsized":"2024-09-17T17:30:09.559751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.521377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089048649999999997,"stat_sum":{"num_bytes":5423104,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":16,"num_read_kb":3128,"num_write":21,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"4"},{"start":"a","length":"3"}]},{"pgid":"3.d","version":"60'48","reported_seq":194,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T17:30:12.403325+0000","last_change":"2024-09-17T17:30:12.403733+0000","last_active":"2024-09-17T17:30:12.403325+0000","last_peered":"2024-09-17T17:30:12.403325+0000","last_clean":"2024-09-17T17:30:12.403325+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:30:12.403325+0000","last_undegraded":"2024-09-17T17:30:12.403325+0000","last_fullsized":"2024-09-17T17:30:12.403325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:59.782056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074316900000000001,"stat_sum":{"num_bytes":5062656,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3609,"num_write":37,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"4"},{"start":"e","length":"1"}]},{"pgid":"3.c","version":"62'79","reported_seq":249,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.444056+0000","last_change":"2024-09-17T17:30:13.438667+0000","last_active":"2024-09-17T17:30:13.444056+0000","last_peered":"2024-09-17T17:30:13.444056+0000","last_clean":"2024-09-17T17:30:13.444056+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:13.444056+0000","last_undegraded":"2024-09-17T17:30:13.444056+0000","last_fullsized":"2024-09-17T17:30:13.444056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.589875+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.017718680000000001,"stat_sum":{"num_bytes":7651328,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":3628,"num_write":56,"num_write_kb":10144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.b","version":"60'12","reported_seq":172,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.427709+0000","last_change":"2024-09-17T17:30:13.428111+0000","last_active":"2024-09-17T17:30:13.427709+0000","last_peered":"2024-09-17T17:30:13.427709+0000","last_clean":"2024-09-17T17:30:13.427709+0000","last_became_active":"2024-09-17T17:29:44.653855+0000","last_became_peered":"2024-09-17T17:29:44.653855+0000","last_unstale":"2024-09-17T17:30:13.427709+0000","last_undegraded":"2024-09-17T17:30:13.427709+0000","last_fullsized":"2024-09-17T17:30:13.427709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076427600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":6718,"num_write":6,"num_write_kb":1456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,10,3,8,7,14],"acting":[13,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.a","version":"62'84","reported_seq":247,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.504012+0000","last_change":"2024-09-17T17:30:13.456791+0000","last_active":"2024-09-17T17:30:13.504012+0000","last_peered":"2024-09-17T17:30:13.504012+0000","last_clean":"2024-09-17T17:30:13.504012+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:13.504012+0000","last_undegraded":"2024-09-17T17:30:13.504012+0000","last_fullsized":"2024-09-17T17:30:13.504012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:48.483980+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0061481139999999997,"stat_sum":{"num_bytes":6651904,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":7941,"num_write":70,"num_write_kb":7456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.9","version":"55'36","reported_seq":173,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.377103+0000","last_change":"2024-09-17T17:30:08.366190+0000","last_active":"2024-09-17T17:30:09.377103+0000","last_peered":"2024-09-17T17:30:09.377103+0000","last_clean":"2024-09-17T17:30:09.377103+0000","last_became_active":"2024-09-17T17:29:48.188156+0000","last_became_peered":"2024-09-17T17:29:48.188156+0000","last_unstale":"2024-09-17T17:30:09.377103+0000","last_undegraded":"2024-09-17T17:30:09.377103+0000","last_fullsized":"2024-09-17T17:30:09.377103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:49.996501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068130399999999998,"stat_sum":{"num_bytes":3309568,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":17,"num_write_kb":3888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,6,10,4,15,9,1],"acting":[12,6,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"5","length":"4"},{"start":"a","length":"3"}]},{"pgid":"3.8","version":"60'17","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.404165+0000","last_change":"2024-09-17T17:30:13.404503+0000","last_active":"2024-09-17T17:30:13.404165+0000","last_peered":"2024-09-17T17:30:13.404165+0000","last_clean":"2024-09-17T17:30:13.404165+0000","last_became_active":"2024-09-17T17:29:48.217378+0000","last_became_peered":"2024-09-17T17:29:48.217378+0000","last_unstale":"2024-09-17T17:30:13.404165+0000","last_undegraded":"2024-09-17T17:30:13.404165+0000","last_fullsized":"2024-09-17T17:30:13.404165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:06.788932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000588022,"stat_sum":{"num_bytes":2031616,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":7599,"num_write":8,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,3],"acting":[6,1,0,4,5,15,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"1.0","version":"25'112","reported_seq":222,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.404052+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:30:13.404052+0000","last_peered":"2024-09-17T17:30:13.404052+0000","last_clean":"2024-09-17T17:30:13.404052+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:13.404052+0000","last_undegraded":"2024-09-17T17:30:13.404052+0000","last_fullsized":"2024-09-17T17:30:13.404052+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":88,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432079+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:30:13.432079+0000","last_peered":"2024-09-17T17:30:13.432079+0000","last_clean":"2024-09-17T17:30:13.432079+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:13.432079+0000","last_undegraded":"2024-09-17T17:30:13.432079+0000","last_fullsized":"2024-09-17T17:30:13.432079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"60'38","reported_seq":186,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432366+0000","last_change":"2024-09-17T17:30:13.432686+0000","last_active":"2024-09-17T17:30:13.432366+0000","last_peered":"2024-09-17T17:30:13.432366+0000","last_clean":"2024-09-17T17:30:13.432366+0000","last_became_active":"2024-09-17T17:29:44.654917+0000","last_became_peered":"2024-09-17T17:29:44.654917+0000","last_unstale":"2024-09-17T17:30:13.432366+0000","last_undegraded":"2024-09-17T17:30:13.432366+0000","last_fullsized":"2024-09-17T17:30:13.432366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:55.544812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063548700000000001,"stat_sum":{"num_bytes":2228224,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,10,11],"acting":[14,12,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.450508+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:30:13.450508+0000","last_peered":"2024-09-17T17:30:13.450508+0000","last_clean":"2024-09-17T17:30:13.450508+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:13.450508+0000","last_undegraded":"2024-09-17T17:30:13.450508+0000","last_fullsized":"2024-09-17T17:30:13.450508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"60'25","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.427834+0000","last_change":"2024-09-17T17:30:13.428271+0000","last_active":"2024-09-17T17:30:13.427834+0000","last_peered":"2024-09-17T17:30:13.427834+0000","last_clean":"2024-09-17T17:30:13.427834+0000","last_became_active":"2024-09-17T17:29:48.191774+0000","last_became_peered":"2024-09-17T17:29:48.191774+0000","last_unstale":"2024-09-17T17:30:13.427834+0000","last_undegraded":"2024-09-17T17:30:13.427834+0000","last_fullsized":"2024-09-17T17:30:13.427834+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:45.076435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000921293,"stat_sum":{"num_bytes":2818048,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2994,"num_write":32,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,4],"acting":[13,11,14,10,12,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432115+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:30:13.432115+0000","last_peered":"2024-09-17T17:30:13.432115+0000","last_clean":"2024-09-17T17:30:13.432115+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:13.432115+0000","last_undegraded":"2024-09-17T17:30:13.432115+0000","last_fullsized":"2024-09-17T17:30:13.432115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"57'46","reported_seq":190,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.377098+0000","last_change":"2024-09-17T17:30:08.383767+0000","last_active":"2024-09-17T17:30:09.377098+0000","last_peered":"2024-09-17T17:30:09.377098+0000","last_clean":"2024-09-17T17:30:09.377098+0000","last_became_active":"2024-09-17T17:29:48.191827+0000","last_became_peered":"2024-09-17T17:29:48.191827+0000","last_unstale":"2024-09-17T17:30:09.377098+0000","last_undegraded":"2024-09-17T17:30:09.377098+0000","last_fullsized":"2024-09-17T17:30:09.377098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:04.392403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018298901999999999,"stat_sum":{"num_bytes":3178496,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":18,"num_read_kb":1334,"num_write":17,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,0,6,14,9],"acting":[12,11,15,0,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"5","length":"4"},{"start":"a","length":"3"}]},{"pgid":"2.2","version":"28'2","reported_seq":89,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.450656+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:13.450656+0000","last_peered":"2024-09-17T17:30:13.450656+0000","last_clean":"2024-09-17T17:30:13.450656+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:13.450656+0000","last_undegraded":"2024-09-17T17:30:13.450656+0000","last_fullsized":"2024-09-17T17:30:13.450656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"62'50","reported_seq":226,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.459596+0000","last_change":"2024-09-17T17:30:13.428694+0000","last_active":"2024-09-17T17:30:13.459596+0000","last_peered":"2024-09-17T17:30:13.459596+0000","last_clean":"2024-09-17T17:30:13.459596+0000","last_became_active":"2024-09-17T17:29:48.420194+0000","last_became_peered":"2024-09-17T17:29:48.420194+0000","last_unstale":"2024-09-17T17:30:13.459596+0000","last_undegraded":"2024-09-17T17:30:13.459596+0000","last_fullsized":"2024-09-17T17:30:13.459596+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.669683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086478900000000003,"stat_sum":{"num_bytes":7012352,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":4338,"num_write":25,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,5,3],"acting":[10,7,1,15,4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.5","version":"40'20","reported_seq":197,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.433460+0000","last_change":"2024-09-17T17:30:13.433759+0000","last_active":"2024-09-17T17:30:13.433460+0000","last_peered":"2024-09-17T17:30:13.433460+0000","last_clean":"2024-09-17T17:30:13.433460+0000","last_became_active":"2024-09-17T17:29:48.422075+0000","last_became_peered":"2024-09-17T17:29:48.422075+0000","last_unstale":"2024-09-17T17:30:13.433460+0000","last_undegraded":"2024-09-17T17:30:13.433460+0000","last_fullsized":"2024-09-17T17:30:13.433460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:56.174154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057357800000000004,"stat_sum":{"num_bytes":1736704,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3370,"num_write":32,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,3,7,1,8],"acting":[15,13,5,3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T17:30:11.387801+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:30:11.387801+0000","last_peered":"2024-09-17T17:30:11.387801+0000","last_clean":"2024-09-17T17:30:11.387801+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:30:11.387801+0000","last_undegraded":"2024-09-17T17:30:11.387801+0000","last_fullsized":"2024-09-17T17:30:11.387801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"60'34","reported_seq":187,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T17:30:11.392213+0000","last_change":"2024-09-17T17:30:11.393345+0000","last_active":"2024-09-17T17:30:11.392213+0000","last_peered":"2024-09-17T17:30:11.392213+0000","last_clean":"2024-09-17T17:30:11.392213+0000","last_became_active":"2024-09-17T17:29:48.188164+0000","last_became_peered":"2024-09-17T17:29:48.188164+0000","last_unstale":"2024-09-17T17:30:11.392213+0000","last_undegraded":"2024-09-17T17:30:11.392213+0000","last_fullsized":"2024-09-17T17:30:11.392213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:02.096250+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0055942199999999996,"stat_sum":{"num_bytes":2965504,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2742,"num_write":18,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,4,6,10,13],"acting":[1,12,9,4,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T17:30:09.559638+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:09.559638+0000","last_peered":"2024-09-17T17:30:09.559638+0000","last_clean":"2024-09-17T17:30:09.559638+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:09.559638+0000","last_undegraded":"2024-09-17T17:30:09.559638+0000","last_fullsized":"2024-09-17T17:30:09.559638+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":160,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.428188+0000","last_change":"2024-09-17T17:30:13.428598+0000","last_active":"2024-09-17T17:30:13.428188+0000","last_peered":"2024-09-17T17:30:13.428188+0000","last_clean":"2024-09-17T17:30:13.428188+0000","last_became_active":"2024-09-17T17:29:44.655172+0000","last_became_peered":"2024-09-17T17:29:44.655172+0000","last_unstale":"2024-09-17T17:30:13.428188+0000","last_undegraded":"2024-09-17T17:30:13.428188+0000","last_fullsized":"2024-09-17T17:30:13.428188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:04.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000803902,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":3,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.424097+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:13.424097+0000","last_peered":"2024-09-17T17:30:13.424097+0000","last_clean":"2024-09-17T17:30:13.424097+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:13.424097+0000","last_undegraded":"2024-09-17T17:30:13.424097+0000","last_fullsized":"2024-09-17T17:30:13.424097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"62'107","reported_seq":274,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.443343+0000","last_change":"2024-09-17T17:30:13.439398+0000","last_active":"2024-09-17T17:30:13.443343+0000","last_peered":"2024-09-17T17:30:13.443343+0000","last_clean":"2024-09-17T17:30:13.443343+0000","last_became_active":"2024-09-17T17:29:48.419866+0000","last_became_peered":"2024-09-17T17:29:48.419866+0000","last_unstale":"2024-09-17T17:30:13.443343+0000","last_undegraded":"2024-09-17T17:30:13.443343+0000","last_fullsized":"2024-09-17T17:30:13.443343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:56.022859+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011268261,"stat_sum":{"num_bytes":7979008,"num_objects":6,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":340,"num_write":49,"num_write_kb":15056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,15,1,7,0],"acting":[13,14,6,15,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.433098+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:13.433098+0000","last_peered":"2024-09-17T17:30:13.433098+0000","last_clean":"2024-09-17T17:30:13.433098+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:13.433098+0000","last_undegraded":"2024-09-17T17:30:13.433098+0000","last_fullsized":"2024-09-17T17:30:13.433098+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":65617920,"num_objects":48,"num_object_clones":6,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":2,"num_read":281,"num_read_kb":48041,"num_write":444,"num_write_kb":82784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106274816,"data_stored":114106368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":695,"ondisk_log_size":695,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39504,"kb_used_data":8308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332336,"statfs":{"total":96636764160,"available":96596312064,"internally_reserved":0,"allocated":8507392,"data_stored":11562174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38880,"kb_used_data":7688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332960,"statfs":{"total":96636764160,"available":96596951040,"internally_reserved":0,"allocated":7872512,"data_stored":7601658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247820,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34960,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336880,"statfs":{"total":96636764160,"available":96600965120,"internally_reserved":0,"allocated":3858432,"data_stored":3923134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313228,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45908,"kb_used_data":10556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325932,"statfs":{"total":96636764160,"available":96589754368,"internally_reserved":0,"allocated":10809344,"data_stored":13471048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34040,"kb_used_data":7008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337800,"statfs":{"total":96636764160,"available":96601907200,"internally_reserved":0,"allocated":7176192,"data_stored":7204346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313229,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36588,"kb_used_data":5392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335252,"statfs":{"total":96636764160,"available":96599298048,"internally_reserved":0,"allocated":5521408,"data_stored":5563801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280525,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":6868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":7032832,"data_stored":7057434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36820,"kb_used_data":9784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":10018816,"data_stored":10382842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42988,"kb_used_data":7636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328852,"statfs":{"total":96636764160,"available":96592744448,"internally_reserved":0,"allocated":7819264,"data_stored":7431755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280525,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5693440,"data_stored":5643789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280524,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":5128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":5251072,"data_stored":5111290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,7,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":6536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":6692864,"data_stored":6729210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":94489280524,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":5042176,"data_stored":4664845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39108,"kb_used_data":7912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332732,"statfs":{"total":96636764160,"available":96596717568,"internally_reserved":0,"allocated":8101888,"data_stored":10579134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247820,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41288,"kb_used_data":10092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330552,"statfs":{"total":96636764160,"available":96594485248,"internally_reserved":0,"allocated":10334208,"data_stored":9943147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6090752,"data_stored":6090752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8114176,"data_stored":8859648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6180864,"data_stored":6180864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7327744,"data_stored":10162176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6963200,"data_stored":6963200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":9682944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4698112,"data_stored":5115904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":3084288,"data_stored":3506176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5713920,"data_stored":5713920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6832128,"data_stored":6832128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8347648,"data_stored":8347648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":7733248,"data_stored":11145216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":10473472,"data_stored":10473472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6766592,"data_stored":6766592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8265728,"data_stored":8265728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:30:15.211 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T17:30:15.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:15.212 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-17T17:30:15.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:30:15.227 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:15.225+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:15.293 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:15.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:15.333 INFO:tasks.rados.rados.0.smithi102.stdout:211: done (2 left) 2024-09-17T17:30:15.333 INFO:tasks.rados.rados.0.smithi102.stdout:213: done (1 left) 2024-09-17T17:30:15.333 INFO:tasks.rados.rados.0.smithi102.stdout:214: done (0 left) 2024-09-17T17:30:15.333 INFO:tasks.rados.rados.0.smithi102.stdout:215: setattr oid 46 current snap is 15 2024-09-17T17:30:15.335 INFO:tasks.rados.rados.0.smithi102.stdout:216: read oid 20 snap -1 2024-09-17T17:30:15.335 INFO:tasks.rados.rados.0.smithi102.stdout:216: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-17T17:30:15.335 INFO:tasks.rados.rados.0.smithi102.stdout:217: snap_create 2024-09-17T17:30:15.342 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 40 (ObjNum 79 snap 8 seq_num 79) dirty exists 2024-09-17T17:30:15.343 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:15.341+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:15.343 INFO:tasks.rados.rados.0.smithi102.stdout:216: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-17T17:30:15.343 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:15.346 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:15.349 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:15.429 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:15.427+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.765677513, 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002820571, 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684890+0000", 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3499000000000001e-05 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685111+0000", 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022111 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685121+0000", 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7070000000000004e-06 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685199+0000", 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8145999999999993e-05 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685209+0000", 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0138e-05 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687657+0000", 2024-09-17T17:30:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024481260000000001 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687696+0000", 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.909e-05 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 39/35 ECSubWrite(tid=50, reqid=client.4566.0:195, at_version=39'6, trim_to=0'0, roll_forward_to=36'4))", 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.758590051999999, 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022546670000000001, 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691963+0000", 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8700000000000002e-07 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692554+0000", 2024-09-17T17:30:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000590721 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692590+0000", 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6350000000000003e-05 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692602+0000", 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692708+0000", 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106835 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.692718+0000", 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694181+0000", 2024-09-17T17:30:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014632149999999999 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.694218+0000", 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6730999999999998e-05 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 50/35 ECSubWrite(tid=89, reqid=client.4566.0:401, at_version=50'26, trim_to=0'0, roll_forward_to=49'24))", 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.096871524999999, 2024-09-17T17:30:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002465409, 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353683+0000", 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2890000000000001e-06 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353717+0000", 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4183000000000001e-05 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353760+0000", 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.316e-05 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353773+0000", 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3037000000000001e-05 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353852+0000", 2024-09-17T17:30:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8621000000000005e-05 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353862+0000", 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0614e-05 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356113+0000", 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022509050000000001 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356147+0000", 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999997e-05 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=94, reqid=client.4566.0:406, at_version=51'30, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.107342171999999, 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003277556, 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343212+0000", 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343225+0000", 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414999999999999e-05 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343249+0000", 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3983999999999999e-05 2024-09-17T17:30:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343254+0000", 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5249999999999996e-06 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343307+0000", 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2167999999999999e-05 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343315+0000", 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8580000000000006e-06 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346453+0000", 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003137908 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346488+0000", 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5021999999999998e-05 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 51/35 ECSubWrite(tid=62, reqid=osd.12.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.105999956, 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025408480000000001, 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344554+0000", 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8100000000000001e-07 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344567+0000", 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.254e-05 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344609+0000", 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2549000000000002e-05 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344619+0000", 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6299999999999993e-06 2024-09-17T17:30:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344683+0000", 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4090999999999993e-05 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344693+0000", 2024-09-17T17:30:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0363000000000001e-05 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347044+0000", 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023509339999999998 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347094+0000", 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.986e-05 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=97, reqid=osd.11.0:97, at_version=51'34, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.105507291, 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024083809999999998, 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345046+0000", 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345065+0000", 2024-09-17T17:30:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8292e-05 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345091+0000", 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6812999999999999e-05 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345100+0000", 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7010000000000002e-06 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345388+0000", 2024-09-17T17:30:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028787499999999999 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.345396+0000", 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2269999999999993e-06 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347422+0000", 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020255170000000001 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347454+0000", 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2443000000000001e-05 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.061035514, 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024942340000000001, 2024-09-17T17:30:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389963+0000", 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044482000000000001 2024-09-17T17:30:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389993+0000", 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.995e-05 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389999+0000", 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7830000000000004e-06 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390051+0000", 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2475999999999997e-05 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390057+0000", 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7540000000000003e-06 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.391979+0000", 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001921835 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.392012+0000", 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2848000000000001e-05 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 56/35 ECSubWrite(tid=57, reqid=osd.10.0:57, at_version=56'41, trim_to=0'0, roll_forward_to=53'39))", 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1152895970000003, 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036979209999999998, 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335264+0000", 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335298+0000", 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3837000000000001e-05 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335654+0000", 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035630699999999999 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335665+0000", 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1620999999999999e-05 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335751+0000", 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5692999999999994e-05 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335763+0000", 2024-09-17T17:30:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140000000000001e-05 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338926+0000", 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031629399999999999 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338961+0000", 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5129999999999997e-05 2024-09-17T17:30:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1124838829999995, 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003185701, 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:15.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338094+0000", 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4428000000000001e-05 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011644700000000001 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338219+0000", 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9020000000000005e-06 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338416+0000", 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196558 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338424+0000", 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.585e-06 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341209+0000", 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027847050000000002 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341255+0000", 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1124426530000004, 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004124801, 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:15.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9600000000000001e-07 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338129+0000", 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8003e-05 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338243+0000", 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114367 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338249+0000", 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3160000000000002e-06 2024-09-17T17:30:15.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338788+0000", 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053876399999999995 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338797+0000", 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-06 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342196+0000", 2024-09-17T17:30:15.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033993830000000002 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342235+0000", 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8921999999999998e-05 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:15.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1123437190000001, 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040538249999999996, 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:15.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.99e-07 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338233+0000", 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3623999999999999e-05 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338275+0000", 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2249000000000002e-05 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338284+0000", 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2609999999999993e-06 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338436+0000", 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152032 2024-09-17T17:30:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338448+0000", 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2469999999999999e-05 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342230+0000", 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037816899999999999 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342263+0000", 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3300000000000003e-05 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1122831079999997, 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029941270000000001, 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338289+0000", 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8340000000000001e-05 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338322+0000", 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3419e-05 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338332+0000", 2024-09-17T17:30:15.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0086e-05 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338873+0000", 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054120000000000004 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338882+0000", 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1329999999999999e-06 2024-09-17T17:30:15.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341233+0000", 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023510699999999998 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1365999999999998e-05 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 57/35 ECSubWrite(tid=140, reqid=osd.6.0:140, at_version=57'15, trim_to=0'0, roll_forward_to=56'13))", 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0840252509999999, 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024870859999999999, 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366550+0000", 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1702e-05 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366615+0000", 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5030999999999995e-05 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366623+0000", 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366683+0000", 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9488999999999999e-05 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366692+0000", 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5089999999999999e-06 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.368974+0000", 2024-09-17T17:30:15.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022823029999999998 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.369015+0000", 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0383000000000003e-05 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.078178276, 2024-09-17T17:30:15.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285746, 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372399+0000", 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4416000000000001e-05 2024-09-17T17:30:15.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372479+0000", 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9556999999999997e-05 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372487+0000", 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7589999999999997e-06 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372561+0000", 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3807000000000007e-05 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372571+0000", 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0532e-05 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375622+0000", 2024-09-17T17:30:15.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030507860000000002 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375661+0000", 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8634000000000003e-05 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.07744851, 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004179131, 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373125+0000", 2024-09-17T17:30:15.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9626000000000001e-05 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373159+0000", 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4190999999999998e-05 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373168+0000", 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6999999999999997e-06 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373237+0000", 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9306000000000004e-05 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373246+0000", 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377245+0000", 2024-09-17T17:30:15.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039987390000000003 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377284+0000", 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8423999999999997e-05 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:15.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0771023919999996, 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048101309999999996, 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1800000000000005e-07 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373476+0000", 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4712999999999999e-05 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373535+0000", 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8536999999999999e-05 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373545+0000", 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0142e-05 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374163+0000", 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061771899999999997 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374172+0000", 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.081e-06 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378233+0000", 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040611509999999998 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378261+0000", 2024-09-17T17:30:15.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000002e-05 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0770523069999998, 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038195389999999998, 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373518+0000", 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373595+0000", 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6206999999999997e-05 2024-09-17T17:30:15.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373601+0000", 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6039999999999998e-06 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373659+0000", 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7442000000000003e-05 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373668+0000", 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2040000000000003e-06 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377300+0000", 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00363182 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377320+0000", 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0598000000000002e-05 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0744507819999995, 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034371530000000001, 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376103+0000", 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376128+0000", 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5109e-05 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376199+0000", 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1783e-05 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376209+0000", 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1710000000000008e-06 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376272+0000", 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749000000000002e-05 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376281+0000", 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:30:15.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379498+0000", 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003216605 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379539+0000", 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1587e-05 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.377735+0000", 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.0728181609999998, 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024431040000000002, 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377735+0000", 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377735+0000", 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377736+0000", 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377756+0000", 2024-09-17T17:30:15.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0415e-05 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377792+0000", 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6139000000000001e-05 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377802+0000", 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6250000000000002e-06 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378477+0000", 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067517399999999996 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378486+0000", 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5030000000000003e-06 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380145+0000", 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016589599999999999 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.380178+0000", 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2526e-05 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 64/35 ECSubWrite(tid=108, reqid=client.4566.0:565, at_version=64'40, trim_to=0'0, roll_forward_to=60'38))", 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.11308139, 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002766442, 2024-09-17T17:30:15.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337646+0000", 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017400099999999999 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337696+0000", 2024-09-17T17:30:15.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0402999999999998e-05 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337712+0000", 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5441999999999999e-05 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337788+0000", 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6283000000000001e-05 2024-09-17T17:30:15.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337842+0000", 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3681999999999999e-05 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.340201+0000", 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023587859999999999 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.340238+0000", 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7404999999999998e-05 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.549 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:15.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:15.552 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:15.549+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:30:15.574 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-17T17:30:15.331116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64078387,"num_objects":45,"num_object_clones":1,"num_object_copies":295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":367,"num_read_kb":48114,"num_write":591,"num_write_kb":85458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":820,"ondisk_log_size":820,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":561696,"kb_used_data":110440,"kb_used_omap":23,"kb_used_meta":450856,"kb_avail":1415015904,"statfs":{"total":1449551462400,"available":1448976285696,"internally_reserved":0,"allocated":113090560,"data_stored":113499815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":461677194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2637824,"num_objects":-25,"num_object_clones":-22,"num_object_copies":-175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":-3,"num_read":11,"num_read_kb":2996,"num_write":79,"num_write_kb":16032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.944010"},"pg_stats":[{"pgid":"3.f","version":"60'56","reported_seq":219,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.421473+0000","last_change":"2024-09-17T17:30:13.421473+0000","last_active":"2024-09-17T17:30:13.421473+0000","last_peered":"2024-09-17T17:30:13.421473+0000","last_clean":"2024-09-17T17:30:13.421473+0000","last_became_active":"2024-09-17T17:29:48.191466+0000","last_became_peered":"2024-09-17T17:29:48.191466+0000","last_unstale":"2024-09-17T17:30:13.421473+0000","last_undegraded":"2024-09-17T17:30:13.421473+0000","last_fullsized":"2024-09-17T17:30:13.421473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.350540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050831599999999998,"stat_sum":{"num_bytes":6979584,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":292,"num_write":40,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,6,13,1],"acting":[11,12,4,14,6,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.e","version":"60'39","reported_seq":199,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T17:30:14.414279+0000","last_change":"2024-09-17T17:30:13.454167+0000","last_active":"2024-09-17T17:30:14.414279+0000","last_peered":"2024-09-17T17:30:14.414279+0000","last_clean":"2024-09-17T17:30:14.414279+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:14.414279+0000","last_undegraded":"2024-09-17T17:30:14.414279+0000","last_fullsized":"2024-09-17T17:30:14.414279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:03.417151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060511899999999999,"stat_sum":{"num_bytes":4358144,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3128,"num_write":21,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.d","version":"60'48","reported_seq":194,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T17:30:12.403325+0000","last_change":"2024-09-17T17:30:12.403733+0000","last_active":"2024-09-17T17:30:12.403325+0000","last_peered":"2024-09-17T17:30:12.403325+0000","last_clean":"2024-09-17T17:30:12.403325+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:30:12.403325+0000","last_undegraded":"2024-09-17T17:30:12.403325+0000","last_fullsized":"2024-09-17T17:30:12.403325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:59.782056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074316900000000001,"stat_sum":{"num_bytes":5062656,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3609,"num_write":37,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"4"},{"start":"e","length":"1"}]},{"pgid":"3.c","version":"62'79","reported_seq":249,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.444056+0000","last_change":"2024-09-17T17:30:13.438667+0000","last_active":"2024-09-17T17:30:13.444056+0000","last_peered":"2024-09-17T17:30:13.444056+0000","last_clean":"2024-09-17T17:30:13.444056+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:13.444056+0000","last_undegraded":"2024-09-17T17:30:13.444056+0000","last_fullsized":"2024-09-17T17:30:13.444056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.589875+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.017718680000000001,"stat_sum":{"num_bytes":7651328,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":3628,"num_write":56,"num_write_kb":10144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.b","version":"60'12","reported_seq":172,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.427709+0000","last_change":"2024-09-17T17:30:13.428111+0000","last_active":"2024-09-17T17:30:13.427709+0000","last_peered":"2024-09-17T17:30:13.427709+0000","last_clean":"2024-09-17T17:30:13.427709+0000","last_became_active":"2024-09-17T17:29:44.653855+0000","last_became_peered":"2024-09-17T17:29:44.653855+0000","last_unstale":"2024-09-17T17:30:13.427709+0000","last_undegraded":"2024-09-17T17:30:13.427709+0000","last_fullsized":"2024-09-17T17:30:13.427709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076427600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":6718,"num_write":6,"num_write_kb":1456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,10,3,8,7,14],"acting":[13,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.a","version":"62'84","reported_seq":247,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.504012+0000","last_change":"2024-09-17T17:30:13.456791+0000","last_active":"2024-09-17T17:30:13.504012+0000","last_peered":"2024-09-17T17:30:13.504012+0000","last_clean":"2024-09-17T17:30:13.504012+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:13.504012+0000","last_undegraded":"2024-09-17T17:30:13.504012+0000","last_fullsized":"2024-09-17T17:30:13.504012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:48.483980+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0061481139999999997,"stat_sum":{"num_bytes":6651904,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":7941,"num_write":70,"num_write_kb":7456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.9","version":"60'38","reported_seq":202,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T17:30:14.421112+0000","last_change":"2024-09-17T17:30:13.404771+0000","last_active":"2024-09-17T17:30:14.421112+0000","last_peered":"2024-09-17T17:30:14.421112+0000","last_clean":"2024-09-17T17:30:14.421112+0000","last_became_active":"2024-09-17T17:29:48.188156+0000","last_became_peered":"2024-09-17T17:29:48.188156+0000","last_unstale":"2024-09-17T17:30:14.421112+0000","last_undegraded":"2024-09-17T17:30:14.421112+0000","last_fullsized":"2024-09-17T17:30:14.421112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:12.646909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074454899999999995,"stat_sum":{"num_bytes":3309568,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":17,"num_write_kb":3888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,6,10,4,15,9,1],"acting":[12,6,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.8","version":"60'17","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.404165+0000","last_change":"2024-09-17T17:30:13.404503+0000","last_active":"2024-09-17T17:30:13.404165+0000","last_peered":"2024-09-17T17:30:13.404165+0000","last_clean":"2024-09-17T17:30:13.404165+0000","last_became_active":"2024-09-17T17:29:48.217378+0000","last_became_peered":"2024-09-17T17:29:48.217378+0000","last_unstale":"2024-09-17T17:30:13.404165+0000","last_undegraded":"2024-09-17T17:30:13.404165+0000","last_fullsized":"2024-09-17T17:30:13.404165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:06.788932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000588022,"stat_sum":{"num_bytes":2031616,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":7599,"num_write":8,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,3],"acting":[6,1,0,4,5,15,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"1.0","version":"25'112","reported_seq":222,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.404052+0000","last_change":"2024-09-17T17:29:29.847841+0000","last_active":"2024-09-17T17:30:13.404052+0000","last_peered":"2024-09-17T17:30:13.404052+0000","last_clean":"2024-09-17T17:30:13.404052+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:13.404052+0000","last_undegraded":"2024-09-17T17:30:13.404052+0000","last_fullsized":"2024-09-17T17:30:13.404052+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:29:28.558396+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:35.955940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":88,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432079+0000","last_change":"2024-09-17T17:29:33.615621+0000","last_active":"2024-09-17T17:30:13.432079+0000","last_peered":"2024-09-17T17:30:13.432079+0000","last_clean":"2024-09-17T17:30:13.432079+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:13.432079+0000","last_undegraded":"2024-09-17T17:30:13.432079+0000","last_fullsized":"2024-09-17T17:30:13.432079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:36.351249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"60'38","reported_seq":186,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432366+0000","last_change":"2024-09-17T17:30:13.432686+0000","last_active":"2024-09-17T17:30:13.432366+0000","last_peered":"2024-09-17T17:30:13.432366+0000","last_clean":"2024-09-17T17:30:13.432366+0000","last_became_active":"2024-09-17T17:29:44.654917+0000","last_became_peered":"2024-09-17T17:29:44.654917+0000","last_unstale":"2024-09-17T17:30:13.432366+0000","last_undegraded":"2024-09-17T17:30:13.432366+0000","last_fullsized":"2024-09-17T17:30:13.432366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:55.544812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063548700000000001,"stat_sum":{"num_bytes":2228224,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,10,11],"acting":[14,12,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.450508+0000","last_change":"2024-09-17T17:29:33.592519+0000","last_active":"2024-09-17T17:30:13.450508+0000","last_peered":"2024-09-17T17:30:13.450508+0000","last_clean":"2024-09-17T17:30:13.450508+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:13.450508+0000","last_undegraded":"2024-09-17T17:30:13.450508+0000","last_fullsized":"2024-09-17T17:30:13.450508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.555279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"60'25","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.427834+0000","last_change":"2024-09-17T17:30:13.428271+0000","last_active":"2024-09-17T17:30:13.427834+0000","last_peered":"2024-09-17T17:30:13.427834+0000","last_clean":"2024-09-17T17:30:13.427834+0000","last_became_active":"2024-09-17T17:29:48.191774+0000","last_became_peered":"2024-09-17T17:29:48.191774+0000","last_unstale":"2024-09-17T17:30:13.427834+0000","last_undegraded":"2024-09-17T17:30:13.427834+0000","last_fullsized":"2024-09-17T17:30:13.427834+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:45.076435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000921293,"stat_sum":{"num_bytes":2818048,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2994,"num_write":32,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,4],"acting":[13,11,14,10,12,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.432115+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:30:13.432115+0000","last_peered":"2024-09-17T17:30:13.432115+0000","last_clean":"2024-09-17T17:30:13.432115+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:13.432115+0000","last_undegraded":"2024-09-17T17:30:13.432115+0000","last_fullsized":"2024-09-17T17:30:13.432115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"60'52","reported_seq":222,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T17:30:14.421416+0000","last_change":"2024-09-17T17:30:13.404993+0000","last_active":"2024-09-17T17:30:14.421416+0000","last_peered":"2024-09-17T17:30:14.421416+0000","last_clean":"2024-09-17T17:30:14.421416+0000","last_became_active":"2024-09-17T17:29:48.191827+0000","last_became_peered":"2024-09-17T17:29:48.191827+0000","last_unstale":"2024-09-17T17:30:14.421416+0000","last_undegraded":"2024-09-17T17:30:14.421416+0000","last_fullsized":"2024-09-17T17:30:14.421416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.775118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086506099999999998,"stat_sum":{"num_bytes":2113536,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":1334,"num_write":17,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,0,6,14,9],"acting":[12,11,15,0,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.2","version":"28'2","reported_seq":89,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.450656+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:13.450656+0000","last_peered":"2024-09-17T17:30:13.450656+0000","last_clean":"2024-09-17T17:30:13.450656+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:13.450656+0000","last_undegraded":"2024-09-17T17:30:13.450656+0000","last_fullsized":"2024-09-17T17:30:13.450656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"62'50","reported_seq":226,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.459596+0000","last_change":"2024-09-17T17:30:13.428694+0000","last_active":"2024-09-17T17:30:13.459596+0000","last_peered":"2024-09-17T17:30:13.459596+0000","last_clean":"2024-09-17T17:30:13.459596+0000","last_became_active":"2024-09-17T17:29:48.420194+0000","last_became_peered":"2024-09-17T17:29:48.420194+0000","last_unstale":"2024-09-17T17:30:13.459596+0000","last_undegraded":"2024-09-17T17:30:13.459596+0000","last_fullsized":"2024-09-17T17:30:13.459596+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:47.669683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086478900000000003,"stat_sum":{"num_bytes":7012352,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":4338,"num_write":25,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,5,3],"acting":[10,7,1,15,4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"3.5","version":"40'20","reported_seq":197,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.433460+0000","last_change":"2024-09-17T17:30:13.433759+0000","last_active":"2024-09-17T17:30:13.433460+0000","last_peered":"2024-09-17T17:30:13.433460+0000","last_clean":"2024-09-17T17:30:13.433460+0000","last_became_active":"2024-09-17T17:29:48.422075+0000","last_became_peered":"2024-09-17T17:29:48.422075+0000","last_unstale":"2024-09-17T17:30:13.433460+0000","last_undegraded":"2024-09-17T17:30:13.433460+0000","last_fullsized":"2024-09-17T17:30:13.433460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:56.174154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057357800000000004,"stat_sum":{"num_bytes":1736704,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3370,"num_write":32,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,3,7,1,8],"acting":[15,13,5,3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T17:30:11.387801+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:30:11.387801+0000","last_peered":"2024-09-17T17:30:11.387801+0000","last_clean":"2024-09-17T17:30:11.387801+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:30:11.387801+0000","last_undegraded":"2024-09-17T17:30:11.387801+0000","last_fullsized":"2024-09-17T17:30:11.387801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:46.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"60'34","reported_seq":187,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T17:30:11.392213+0000","last_change":"2024-09-17T17:30:11.393345+0000","last_active":"2024-09-17T17:30:11.392213+0000","last_peered":"2024-09-17T17:30:11.392213+0000","last_clean":"2024-09-17T17:30:11.392213+0000","last_became_active":"2024-09-17T17:29:48.188164+0000","last_became_peered":"2024-09-17T17:29:48.188164+0000","last_unstale":"2024-09-17T17:30:11.392213+0000","last_undegraded":"2024-09-17T17:30:11.392213+0000","last_fullsized":"2024-09-17T17:30:11.392213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:02.096250+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0055942199999999996,"stat_sum":{"num_bytes":2965504,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2742,"num_write":18,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,4,6,10,13],"acting":[1,12,9,4,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T17:30:14.414185+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:14.414185+0000","last_peered":"2024-09-17T17:30:14.414185+0000","last_clean":"2024-09-17T17:30:14.414185+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:14.414185+0000","last_undegraded":"2024-09-17T17:30:14.414185+0000","last_fullsized":"2024-09-17T17:30:14.414185+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":160,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.428188+0000","last_change":"2024-09-17T17:30:13.428598+0000","last_active":"2024-09-17T17:30:13.428188+0000","last_peered":"2024-09-17T17:30:13.428188+0000","last_clean":"2024-09-17T17:30:13.428188+0000","last_became_active":"2024-09-17T17:29:44.655172+0000","last_became_peered":"2024-09-17T17:29:44.655172+0000","last_unstale":"2024-09-17T17:30:13.428188+0000","last_undegraded":"2024-09-17T17:30:13.428188+0000","last_fullsized":"2024-09-17T17:30:13.428188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:04.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000803902,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":3,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.424097+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:13.424097+0000","last_peered":"2024-09-17T17:30:13.424097+0000","last_clean":"2024-09-17T17:30:13.424097+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:13.424097+0000","last_undegraded":"2024-09-17T17:30:13.424097+0000","last_fullsized":"2024-09-17T17:30:13.424097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:30.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"62'107","reported_seq":274,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.443343+0000","last_change":"2024-09-17T17:30:13.439398+0000","last_active":"2024-09-17T17:30:13.443343+0000","last_peered":"2024-09-17T17:30:13.443343+0000","last_clean":"2024-09-17T17:30:13.443343+0000","last_became_active":"2024-09-17T17:29:48.419866+0000","last_became_peered":"2024-09-17T17:29:48.419866+0000","last_unstale":"2024-09-17T17:30:13.443343+0000","last_undegraded":"2024-09-17T17:30:13.443343+0000","last_fullsized":"2024-09-17T17:30:13.443343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:56.022859+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011268261,"stat_sum":{"num_bytes":7979008,"num_objects":6,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":340,"num_write":49,"num_write_kb":15056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,15,1,7,0],"acting":[13,14,6,15,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T17:30:13.433098+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:13.433098+0000","last_peered":"2024-09-17T17:30:13.433098+0000","last_clean":"2024-09-17T17:30:13.433098+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:13.433098+0000","last_undegraded":"2024-09-17T17:30:13.433098+0000","last_fullsized":"2024-09-17T17:30:13.433098+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:44.173835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":63488000,"num_objects":41,"num_object_clones":1,"num_object_copies":287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":281,"num_read_kb":48041,"num_write":444,"num_write_kb":82784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":109371392,"data_stored":110608384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":705,"ondisk_log_size":705,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40544,"kb_used_data":9352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331296,"statfs":{"total":96636764160,"available":96595247104,"internally_reserved":0,"allocated":9576448,"data_stored":9188939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38880,"kb_used_data":7688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332960,"statfs":{"total":96636764160,"available":96596951040,"internally_reserved":0,"allocated":7872512,"data_stored":7601658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247821,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35280,"kb_used_data":4092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336560,"statfs":{"total":96636764160,"available":96600637440,"internally_reserved":0,"allocated":4190208,"data_stored":3802699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313228,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45908,"kb_used_data":10556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325932,"statfs":{"total":96636764160,"available":96589754368,"internally_reserved":0,"allocated":10809344,"data_stored":13471048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34040,"kb_used_data":7008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337800,"statfs":{"total":96636764160,"available":96601907200,"internally_reserved":0,"allocated":7176192,"data_stored":7204346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313229,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36588,"kb_used_data":5392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335252,"statfs":{"total":96636764160,"available":96599298048,"internally_reserved":0,"allocated":5521408,"data_stored":5563801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280525,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":6868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":7032832,"data_stored":7057434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36820,"kb_used_data":9784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":10018816,"data_stored":10382842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42988,"kb_used_data":7636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328852,"statfs":{"total":96636764160,"available":96592744448,"internally_reserved":0,"allocated":7819264,"data_stored":7431755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280525,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5693440,"data_stored":5643789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280524,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":5128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":5251072,"data_stored":5111290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,7,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":6536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":6692864,"data_stored":6729210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":94489280524,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":5042176,"data_stored":4664845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41020,"kb_used_data":9824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330820,"statfs":{"total":96636764160,"available":96594759680,"internally_reserved":0,"allocated":10059776,"data_stored":9703012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247820,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41288,"kb_used_data":10092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330552,"statfs":{"total":96636764160,"available":96594485248,"internally_reserved":0,"allocated":10334208,"data_stored":9943147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6090752,"data_stored":6090752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8114176,"data_stored":8859648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6180864,"data_stored":6180864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9187328,"data_stored":9224192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6963200,"data_stored":6963200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9682944,"data_stored":9682944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4698112,"data_stored":5115904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":3334144,"data_stored":3334144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5713920,"data_stored":5713920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6832128,"data_stored":6832128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8347648,"data_stored":8347648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":8720384,"data_stored":8757248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":10473472,"data_stored":10473472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6766592,"data_stored":6766592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8265728,"data_stored":8265728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:30:15.576 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-17T17:30:15.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-09-17T17:30:15.651 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:15.650+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:15.660 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:15.661 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:15.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:15.752 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:15.751+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:15.778 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:15.778 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:15.778 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:15.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:15.854 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:15.852+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:15.897 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:15.897 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:15.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:15.897 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:15.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:15.898 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:15.898 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.209008658999998, 2024-09-17T17:30:15.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:30:15.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:15.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:15.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:15.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:30:15.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:30:15.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:30:15.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:30:15.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:30:15.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:30:15.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:30:15.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:30:15.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:30:15.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:30:15.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:30:15.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:30:15.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:30:15.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:30:15.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:30:15.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.5s6 39/35 ECSubWrite(tid=20, reqid=client.4566.0:191, at_version=39'17, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.203474324999998, 2024-09-17T17:30:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001787521, 2024-09-17T17:30:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.911 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:15.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690191+0000", 2024-09-17T17:30:15.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690192+0000", 2024-09-17T17:30:15.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T17:30:15.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690618+0000", 2024-09-17T17:30:15.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042645600000000002 2024-09-17T17:30:15.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690656+0000", 2024-09-17T17:30:15.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8214999999999999e-05 2024-09-17T17:30:15.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690666+0000", 2024-09-17T17:30:15.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5419999999999994e-06 2024-09-17T17:30:15.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690744+0000", 2024-09-17T17:30:15.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8418999999999996e-05 2024-09-17T17:30:15.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.690754+0000", 2024-09-17T17:30:15.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9329999999999996e-06 2024-09-17T17:30:15.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691939+0000", 2024-09-17T17:30:15.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001184198 2024-09-17T17:30:15.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.691978+0000", 2024-09-17T17:30:15.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9845999999999998e-05 2024-09-17T17:30:15.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:15.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:15.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.19547249, 2024-09-17T17:30:15.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:30:15.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:15.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:15.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:15.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:15.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:30:15.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:15.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:15.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:30:15.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:30:15.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:30:15.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:30:15.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:30:15.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:30:15.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:30:15.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:30:15.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:30:15.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:30:15.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:15.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:30:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:30:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:30:15.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:30:15.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:30:15.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:30:15.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:15.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:30:15.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:30:15.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:30:15.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:30:15.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:15.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:15.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.193246631000001, 2024-09-17T17:30:15.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:30:15.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:15.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:15.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:15.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:15.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:30:15.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:15.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:15.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:30:15.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:30:15.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:30:15.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:30:15.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:30:15.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:30:15.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:30:15.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:15.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:30:15.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:30:15.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:15.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:30:15.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:30:15.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:30:15.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:30:15.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:30:15.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:30:15.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:15.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:30:15.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:30:15.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:30:15.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.192822551999999, 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:30:15.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:15.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:30:15.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.192728249999998, 2024-09-17T17:30:15.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:15.962 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.192103286999998, 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015435340000000001, 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:15.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701562+0000", 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701563+0000", 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.155e-06 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701569+0000", 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701604+0000", 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5148999999999998e-05 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701612+0000", 2024-09-17T17:30:15.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9559999999999997e-06 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703084+0000", 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014716270000000001 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703105+0000", 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1766000000000001e-05 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 39/32 ECSubWriteReply(tid=15, last_complete=39'10, committed=1, applied=1))", 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.191865999000001, 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001340387, 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:15.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701799+0000", 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701800+0000", 2024-09-17T17:30:15.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2900000000000002e-07 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701806+0000", 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5099999999999998e-06 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701957+0000", 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000151175 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701965+0000", 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703127+0000", 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011621019999999999 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703140+0000", 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.295e-05 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.7s1 39/32 ECSubWrite(tid=57, reqid=client.4566.0:182, at_version=39'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.190233026000001, 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00185337, 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703432+0000", 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703433+0000", 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4300000000000002e-07 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704005+0000", 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057213000000000001 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704034+0000", 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8662000000000001e-05 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704039+0000", 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1170000000000003e-06 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704101+0000", 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1818999999999999e-05 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.704111+0000", 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0115000000000001e-05 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705248+0000", 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011371420000000001 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705286+0000", 2024-09-17T17:30:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7542000000000001e-05 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=35, reqid=osd.3.0:35, at_version=43'18, trim_to=0'0, roll_forward_to=42'16))", 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.19627749, 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001758098, 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697388+0000", 2024-09-17T17:30:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697411+0000", 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2937000000000001e-05 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697449+0000", 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519000000000002e-05 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697458+0000", 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697532+0000", 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4147999999999995e-05 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697542+0000", 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.893e-06 2024-09-17T17:30:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699107+0000", 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015656050000000001 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699146+0000", 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8525000000000001e-05 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.bs4 43/32 ECSubWrite(tid=92, reqid=osd.13.0:92, at_version=43'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.195788559, 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019485469999999999, 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697917+0000", 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9153999999999998e-05 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697948+0000", 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1538000000000002e-05 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697957+0000", 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698771+0000", 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081437600000000001 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698781+0000", 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9590000000000007e-06 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699776+0000", 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099482900000000003 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699825+0000", 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9280000000000003e-05 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 43/32 ECSubWrite(tid=37, reqid=client.4566.0:303, at_version=43'20, trim_to=0'0, roll_forward_to=43'18))", 2024-09-17T17:30:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.926971988999998, 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013508630000000001, 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966693+0000", 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966695+0000", 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8390000000000001e-06 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.966972+0000", 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027734400000000001 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967017+0000", 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4558000000000003e-05 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967027+0000", 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0431999999999999e-05 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967083+0000", 2024-09-17T17:30:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5686999999999997e-05 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.967090+0000", 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.861e-06 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.968020+0000", 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093037199999999995 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.968044+0000", 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3770000000000001e-05 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=54, reqid=client.4566.0:390, at_version=50'32, trim_to=0'0, roll_forward_to=48'30))", 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.557442956999999, 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001333426, 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336222+0000", 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336223+0000", 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336344+0000", 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121086 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336381+0000", 2024-09-17T17:30:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6823000000000003e-05 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336391+0000", 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0725e-05 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336445+0000", 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3505999999999999e-05 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.336452+0000", 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4749999999999996e-06 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.337528+0000", 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010753760000000001 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.337556+0000", 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7937999999999999e-05 2024-09-17T17:30:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=55, reqid=client.4566.0:394, at_version=50'34, trim_to=0'0, roll_forward_to=48'30))", 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.55305008, 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014379869999999999, 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340615+0000", 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340617+0000", 2024-09-17T17:30:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340887+0000", 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027011199999999998 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340937+0000", 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0489999999999999e-05 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.340949+0000", 2024-09-17T17:30:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2054e-05 2024-09-17T17:30:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.341016+0000", 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7136000000000002e-05 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.341026+0000", 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.342022+0000", 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099594500000000003 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.342053+0000", 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1619999999999999e-05 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 50/32 ECSubWrite(tid=56, reqid=client.4566.0:395, at_version=50'35, trim_to=0'0, roll_forward_to=50'32))", 2024-09-17T17:30:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.549553619999999, 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013727380000000001, 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344112+0000", 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344113+0000", 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344333+0000", 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022061300000000001 2024-09-17T17:30:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344418+0000", 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4672000000000005e-05 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344426+0000", 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.79e-06 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344496+0000", 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0178000000000005e-05 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.344506+0000", 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0299000000000001e-05 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.345448+0000", 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094149200000000004 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.345484+0000", 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6652000000000001e-05 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 51/32 ECSubWrite(tid=61, reqid=osd.3.0:61, at_version=51'39, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.553474103999999, 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00196635, 2024-09-17T17:30:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340192+0000", 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340213+0000", 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.145e-05 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340244+0000", 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1063999999999999e-05 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340255+0000", 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1066e-05 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8850000000000007e-05 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340334+0000", 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0081999999999999e-05 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.342124+0000", 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001789646 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.342157+0000", 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3788999999999998e-05 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 52/32 ECSubWrite(tid=66, reqid=osd.3.0:66, at_version=52'45, trim_to=0'0, roll_forward_to=51'43))", 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.357940+0000", 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.535725181, 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001359168, 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357940+0000", 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357940+0000", 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357941+0000", 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2999999999999999e-07 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.357969+0000", 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7698000000000001e-05 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358012+0000", 2024-09-17T17:30:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3310999999999999e-05 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358022+0000", 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0434999999999999e-05 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358126+0000", 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103191 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.358137+0000", 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0907000000000001e-05 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.359257+0000", 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001120977 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.359299+0000", 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1718999999999998e-05 2024-09-17T17:30:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 56/32 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=56'59, trim_to=0'0, roll_forward_to=53'55))", 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.347958+0000", 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5457076559999994, 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013959630000000001, 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347958+0000", 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347958+0000", 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347959+0000", 2024-09-17T17:30:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5799999999999999e-06 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347988+0000", 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9329999999999999e-05 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348037+0000", 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8995000000000001e-05 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348048+0000", 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1012999999999999e-05 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348134+0000", 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-05 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348144+0000", 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0251999999999999e-05 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349315+0000", 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011705629999999999 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349354+0000", 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8810000000000003e-05 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=82, reqid=osd.3.0:82, at_version=57'61, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5223903960000005, 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035036519999999999, 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-07 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371301+0000", 2024-09-17T17:30:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6461e-05 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372066+0000", 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076444900000000005 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372076+0000", 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372145+0000", 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9586999999999996e-05 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372155+0000", 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0054999999999999e-05 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374734+0000", 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790520000000001 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374778+0000", 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4042000000000003e-05 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=57'63, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.5203010990000001, 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026238189999999999, 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373365+0000", 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0640000000000001e-06 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373399+0000", 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3757000000000002e-05 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373439+0000", 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0028999999999999e-05 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373469+0000", 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0297000000000001e-05 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373543+0000", 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4202999999999994e-05 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373574+0000", 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0895999999999997e-05 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375923+0000", 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023489689999999998 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375988+0000", 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4603999999999993e-05 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.030 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:16.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:16.033 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:16.030+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:16.134 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:16.131+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:16.180 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:16.181 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:16.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:16.234 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:16.232+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:16.296 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:16.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:16.296 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:16.296 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:16.296 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:16.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 3 pg_num to 26 2024-09-17T17:30:16.335 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:16.333+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:16.336 INFO:tasks.rados.rados.0.smithi102.stdout:215: done (2 left) 2024-09-17T17:30:16.336 INFO:tasks.rados.rados.0.smithi102.stdout:216: done (1 left) 2024-09-17T17:30:16.336 INFO:tasks.rados.rados.0.smithi102.stdout:217: done (0 left) 2024-09-17T17:30:16.336 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 5 current snap is 16 2024-09-17T17:30:16.336 INFO:tasks.rados.rados.0.smithi102.stdout:218: seq_num 107 ranges {0=638976,638976=458752,1097728=704512} 2024-09-17T17:30:16.339 INFO:tasks.rados.rados.0.smithi102.stdout:218: writing smithi10238019-5 from 0 to 638976 tid 1 2024-09-17T17:30:16.342 INFO:tasks.rados.rados.0.smithi102.stdout:218: writing smithi10238019-5 from 638976 to 1097728 tid 2 2024-09-17T17:30:16.347 INFO:tasks.rados.rados.0.smithi102.stdout:218: writing smithi10238019-5 from 1097728 to 1802240 tid 3 2024-09-17T17:30:16.347 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 38 current snap is 16 2024-09-17T17:30:16.348 INFO:tasks.rados.rados.0.smithi102.stdout:219: seq_num 108 ranges {622592=655360,1277952=606208,1884160=475136,2359296=81920} 2024-09-17T17:30:16.353 INFO:tasks.rados.rados.0.smithi102.stdout:219: writing smithi10238019-38 from 622592 to 1277952 tid 1 2024-09-17T17:30:16.357 INFO:tasks.rados.rados.0.smithi102.stdout:219: writing smithi10238019-38 from 1277952 to 1884160 tid 2 2024-09-17T17:30:16.360 INFO:tasks.rados.rados.0.smithi102.stdout:219: writing smithi10238019-38 from 1884160 to 2359296 tid 3 2024-09-17T17:30:16.360 INFO:tasks.rados.rados.0.smithi102.stdout:219: writing smithi10238019-38 from 2359296 to 2441216 tid 4 2024-09-17T17:30:16.360 INFO:tasks.rados.rados.0.smithi102.stdout:220: rmattr oid 27 current snap is 16 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:220: done (2 left) 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:221: setattr oid 31 current snap is 16 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: finishing write tid 1 to smithi10238019-5 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: finishing write tid 2 to smithi10238019-5 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: finishing write tid 3 to smithi10238019-5 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: finishing write tid 4 to smithi10238019-5 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: finishing write tid 5 to smithi10238019-5 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: oid 5 updating version 0 to 42 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: oid 5 updating version 42 to 43 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: oid 5 updating version 43 to 44 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: oid 5 version 44 is already newer than 41 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 5 v 44 (ObjNum 107 snap 16 seq_num 107) dirty exists 2024-09-17T17:30:16.361 INFO:tasks.rados.rados.0.smithi102.stdout:218: left oid 5 (ObjNum 107 snap 16 seq_num 107) 2024-09-17T17:30:16.363 INFO:tasks.rados.rados.0.smithi102.stdout:218: done (2 left) 2024-09-17T17:30:16.363 INFO:tasks.rados.rados.0.smithi102.stdout:222: copy_from oid 1 from oid 19 current snap is 16 2024-09-17T17:30:16.363 INFO:tasks.rados.rados.0.smithi102.stdout:223: delete oid 34 current snap is 16 2024-09-17T17:30:16.363 INFO:tasks.rados.rados.0.smithi102.stdout:219: finishing write tid 1 to smithi10238019-38 2024-09-17T17:30:16.364 INFO:tasks.rados.rados.0.smithi102.stdout:219: finishing write tid 2 to smithi10238019-38 2024-09-17T17:30:16.366 INFO:tasks.rados.rados.0.smithi102.stdout:219: finishing write tid 3 to smithi10238019-38 2024-09-17T17:30:16.366 INFO:tasks.rados.rados.0.smithi102.stdout:219: finishing write tid 4 to smithi10238019-38 2024-09-17T17:30:16.367 INFO:tasks.rados.rados.0.smithi102.stdout:219: finishing write tid 5 to smithi10238019-38 2024-09-17T17:30:16.367 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 40 (ObjNum 65 snap 0 seq_num 1465409113) dirty exists 2024-09-17T17:30:16.369 INFO:tasks.rados.rados.0.smithi102.stdout:219: finishing write tid 6 to smithi10238019-38 2024-09-17T17:30:16.369 INFO:tasks.rados.rados.0.smithi102.stdout:219: oid 38 updating version 0 to 113 2024-09-17T17:30:16.369 INFO:tasks.rados.rados.0.smithi102.stdout:219: oid 38 version 113 is already newer than 112 2024-09-17T17:30:16.369 INFO:tasks.rados.rados.0.smithi102.stdout:219: oid 38 version 113 is already newer than 110 2024-09-17T17:30:16.369 INFO:tasks.rados.rados.0.smithi102.stdout:219: oid 38 version 113 is already newer than 111 2024-09-17T17:30:16.369 INFO:tasks.rados.rados.0.smithi102.stdout:219: oid 38 version 113 is already newer than 109 2024-09-17T17:30:16.370 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 38 v 113 (ObjNum 108 snap 16 seq_num 108) dirty exists 2024-09-17T17:30:16.370 INFO:tasks.rados.rados.0.smithi102.stdout:219: left oid 38 (ObjNum 108 snap 16 seq_num 108) 2024-09-17T17:30:16.372 INFO:tasks.rados.rados.0.smithi102.stdout:219: done (3 left) 2024-09-17T17:30:16.372 INFO:tasks.rados.rados.0.smithi102.stdout:221: done (2 left) 2024-09-17T17:30:16.373 INFO:tasks.rados.rados.0.smithi102.stdout:223: done (1 left) 2024-09-17T17:30:16.373 INFO:tasks.rados.rados.0.smithi102.stdout:224: read oid 12 snap -1 2024-09-17T17:30:16.373 INFO:tasks.rados.rados.0.smithi102.stdout:224: expect deleted 2024-09-17T17:30:16.373 INFO:tasks.rados.rados.0.smithi102.stdout:225: snap_create 2024-09-17T17:30:16.381 INFO:tasks.rados.rados.0.smithi102.stdout:222: finishing copy_from racing read to smithi10238019-1 2024-09-17T17:30:16.382 INFO:tasks.rados.rados.0.smithi102.stdout:222: finishing copy_from to smithi10238019-1 2024-09-17T17:30:16.382 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 1 v 5 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-17T17:30:16.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:16.422 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.735689522000001, 2024-09-17T17:30:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:30:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:30:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:30:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:30:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:30:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:30:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:30:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:30:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:30:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:30:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:30:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:30:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:30:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:30:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:30:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:30:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:30:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:211 3.9s0 3:96cbaefd:::smithi10238019-46:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.708200652999999, 2024-09-17T17:30:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068972499999999997, 2024-09-17T17:30:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 211 2024-09-17T17:30:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710514+0000", 2024-09-17T17:30:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0400000000000002e-07 2024-09-17T17:30:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710516+0000", 2024-09-17T17:30:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5269999999999999e-06 2024-09-17T17:30:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710520+0000", 2024-09-17T17:30:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4059999999999998e-06 2024-09-17T17:30:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.710524+0000", 2024-09-17T17:30:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.518e-06 2024-09-17T17:30:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.712807+0000", 2024-09-17T17:30:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002283322 2024-09-17T17:30:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713024+0000", 2024-09-17T17:30:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216629 2024-09-17T17:30:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.713484+0000", 2024-09-17T17:30:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046035200000000003 2024-09-17T17:30:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.714391+0000", 2024-09-17T17:30:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090667600000000003 2024-09-17T17:30:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717343+0000", 2024-09-17T17:30:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002952863 2024-09-17T17:30:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.717411+0000", 2024-09-17T17:30:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7652999999999998e-05 2024-09-17T17:30:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:30:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.704736299, 2024-09-17T17:30:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:30:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:30:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:30:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:30:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:30:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:30:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:30:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:30:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:30:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:30:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:30:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:30:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.672434656, 2024-09-17T17:30:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:30:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:30:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:30:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:30:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:30:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:30:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.670510153999999, 2024-09-17T17:30:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:399 3.0s0 3:00c2d3b1:::smithi10238019-29:head [copy-from ver 2] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.075451845, 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01110614, 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 399 2024-09-17T17:30:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343263+0000", 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343265+0000", 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343273+0000", 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343291+0000", 2024-09-17T17:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7764000000000001e-05 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343299+0000", 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1629999999999996e-06 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343366+0000", 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6398999999999999e-05 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343443+0000", 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7731000000000005e-05 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.352245+0000", 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088010680000000004 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353574+0000", 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013293109999999999 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.354245+0000", 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067153099999999995 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.354369+0000", 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012345600000000001 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:400 3.0s0 3:00c2d3b1:::smithi10238019-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.075396434, 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010156996, 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 400 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343318+0000", 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343319+0000", 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343321+0000", 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5689999999999999e-06 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343335+0000", 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3541e-05 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343343+0000", 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2740000000000007e-06 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343584+0000", 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241179 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.343618+0000", 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4147999999999999e-05 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353248+0000", 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096298000000000009 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353357+0000", 2024-09-17T17:30:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109281 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353475+0000", 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011772 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:410 3.9s0 3:93d1baed:::smithi10238019-5:head [copy-from ver 8] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.080613960000001, 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017989010999999999, 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 410 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338102+0000", 2024-09-17T17:30:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5349999999999999e-06 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338107+0000", 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000003e-06 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338121+0000", 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3743e-05 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338129+0000", 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4209999999999995e-06 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8885000000000003e-05 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338278+0000", 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0192999999999995e-05 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.349495+0000", 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011217173 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350828+0000", 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013328700000000001 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356035+0000", 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052068000000000001 2024-09-17T17:30:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356090+0000", 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4731000000000002e-05 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:411 3.9s0 3:93d1baed:::smithi10238019-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.080562961, 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012330081, 2024-09-17T17:30:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 411 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338155+0000", 2024-09-17T17:30:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0580000000000002e-06 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338164+0000", 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338170+0000", 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2809999999999997e-06 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338459+0000", 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028850300000000001 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338502+0000", 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3068000000000002e-05 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350266+0000", 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011764431000000001 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350370+0000", 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010380200000000001 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350482+0000", 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011173800000000001 2024-09-17T17:30:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 52/35 ECSubWriteReply(tid=76, last_complete=52'24, committed=1, applied=1))", 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.009162335999999, 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.18030790999999999, 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409553+0000", 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:30:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409560+0000", 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589756+0000", 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.180196197 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589765+0000", 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589834+0000", 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9301000000000006e-05 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589860+0000", 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6058999999999999e-05 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:477 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~802816] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0766494879999993, 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014281265, 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 477 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342067+0000", 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.906e-06 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343566+0000", 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001499521 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343581+0000", 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4949e-05 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343590+0000", 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1529999999999997e-06 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343749+0000", 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343862+0000", 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011281399999999999 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.345103+0000", 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001240718 2024-09-17T17:30:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346311+0000", 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012080210000000001 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356249+0000", 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099386809999999996 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356346+0000", 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7062e-05 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:478 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~638976] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.346105+0000", 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0726095440000005, 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010395823, 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 478 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346105+0000", 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346105+0000", 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346107+0000", 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.911e-06 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346705+0000", 2024-09-17T17:30:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059845199999999997 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346723+0000", 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8019000000000001e-05 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346733+0000", 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.781e-06 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346777+0000", 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3535999999999998e-05 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346853+0000", 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6148000000000003e-05 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.347517+0000", 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066432999999999998 2024-09-17T17:30:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351003+0000", 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034862750000000001 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356429+0000", 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005425933 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356501+0000", 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1438000000000003e-05 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:479 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~131072] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.070636812, 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017167436000000001, 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 479 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348079+0000", 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.719e-06 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348181+0000", 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101258 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348196+0000", 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5423000000000001e-05 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348204+0000", 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348898+0000", 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069432499999999998 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349042+0000", 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143848 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349752+0000", 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071013599999999999 2024-09-17T17:30:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351054+0000", 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301618 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365180+0000", 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014125511 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365245+0000", 2024-09-17T17:30:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5647000000000004e-05 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:480 3.0s0 3:00c2d3b1:::smithi10238019-29:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0704944439999995, 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017175950999999998, 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 480 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348221+0000", 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348225+0000", 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969999999999998e-06 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348235+0000", 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312000000000001e-05 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348241+0000", 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340000000000002e-06 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350222+0000", 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019816209999999998 2024-09-17T17:30:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350336+0000", 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113779 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350754+0000", 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041740400000000002 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351686+0000", 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093241600000000004 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365336+0000", 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013650254000000001 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365396+0000", 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9704000000000003e-05 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:481 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0704633619999999, 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018899091, 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 481 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348252+0000", 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348255+0000", 2024-09-17T17:30:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6720000000000002e-06 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348265+0000", 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.132e-06 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348271+0000", 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351236+0000", 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029650309999999999 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351333+0000", 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7449999999999997e-05 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365440+0000", 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106966 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365519+0000", 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8703999999999997e-05 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.367150+0000", 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001631354 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0461413529999994, 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086831600000000005, 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372573+0000", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372590+0000", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6637999999999999e-05 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373245+0000", 2024-09-17T17:30:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00065489300000000003 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373254+0000", 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2259999999999994e-06 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373321+0000", 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6711999999999994e-05 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373330+0000", 2024-09-17T17:30:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9360000000000005e-06 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381218+0000", 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078879850000000001 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381256+0000", 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8483999999999999e-05 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs1 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.374031+0000", 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0446837109999993, 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088084159999999995, 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374031+0000", 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374031+0000", 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374032+0000", 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374055+0000", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2731999999999999e-05 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374105+0000", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0405000000000003e-05 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374114+0000", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9539999999999993e-06 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374812+0000", 2024-09-17T17:30:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069854100000000001 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374821+0000", 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5879999999999998e-06 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382816+0000", 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079951710000000006 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382839+0000", 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3023999999999998e-05 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.375339+0000", 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0433750620000009, 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075275389999999998, 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375339+0000", 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375339+0000", 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375340+0000", 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2399999999999999e-07 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375362+0000", 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1812999999999999e-05 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375397+0000", 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5079000000000001e-05 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375405+0000", 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7600000000000008e-06 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375538+0000", 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132693 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375549+0000", 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0624e-05 2024-09-17T17:30:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382849+0000", 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072999470000000002 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.382867+0000", 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8399000000000001e-05 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:567 3.0s0 3:0ff8386e:::smithi10238019-20:head [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:15.334664+0000", 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.084050736, 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069379649999999999, 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 567 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334664+0000", 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334664+0000", 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334664+0000", 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.03e-07 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334666+0000", 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9889999999999999e-06 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334669+0000", 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3869999999999999e-06 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334672+0000", 2024-09-17T17:30:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.954e-06 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334764+0000", 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1137999999999994e-05 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334771+0000", 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9030000000000004e-06 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337039+0000", 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022674679999999999 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337137+0000", 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7932999999999999e-05 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.341602+0000", 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00446489 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:568 3.0s0 3:0ff8386e:::smithi10238019-20:head [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:15.334693+0000", 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0840217809999999, 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007336029, 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 568 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334693+0000", 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334693+0000", 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334693+0000", 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2000000000000001e-07 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334696+0000", 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334699+0000", 2024-09-17T17:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7929999999999999e-06 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334702+0000", 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.852e-06 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334798+0000", 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6175000000000007e-05 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.334804+0000", 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8769999999999998e-06 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.338219+0000", 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034150080000000002 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.338305+0000", 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5450000000000003e-05 2024-09-17T17:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.342029+0000", 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037240189999999999 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:16.557 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:16.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:30:16.559 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:16.560 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:16.561 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:16.561 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:16.561 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:16.557+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:16.580 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:16.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:16.624 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:16.624 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:16.646 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:16.660 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:16.658+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:16.676 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:16.676 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:16.683 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:16.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:16.705 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:16.726 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:16.747 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:16.761 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:16.759+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:16.769 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:16.790 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:16.811 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:16.832 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:16.832 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:16.833 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:16.835+0000 7f50f37fe640 0 -- 172.21.15.7:0/3247067515 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f50f421f300 msgr2=0x7f50f42216f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:16.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:16.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:16.861 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:16.859+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:16.880 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:16.880 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:16.949 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:16.950 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:16.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:17.002 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:17.000+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:17.063 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:17.060+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:17.065 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.066 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:17.163 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:17.161+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:17.220 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.220 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:17.264 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:17.263+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:17.337 INFO:tasks.rados.rados.0.smithi102.stdout:222: done (2 left) 2024-09-17T17:30:17.337 INFO:tasks.rados.rados.0.smithi102.stdout:224: done (1 left) 2024-09-17T17:30:17.337 INFO:tasks.rados.rados.0.smithi102.stdout:225: done (0 left) 2024-09-17T17:30:17.337 INFO:tasks.rados.rados.0.smithi102.stdout:226: copy_from oid 40 from oid 13 current snap is 17 2024-09-17T17:30:17.338 INFO:tasks.rados.rados.0.smithi102.stdout:227: copy_from oid 38 from oid 48 current snap is 17 2024-09-17T17:30:17.338 INFO:tasks.rados.rados.0.smithi102.stdout:228: read oid 29 snap -1 2024-09-17T17:30:17.338 INFO:tasks.rados.rados.0.smithi102.stdout:228: expect (ObjNum 93 snap 11 seq_num 93) 2024-09-17T17:30:17.338 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 12 current snap is 17 2024-09-17T17:30:17.338 INFO:tasks.rados.rados.0.smithi102.stdout:229: seq_num 110 ranges {0=688128,688128=507904,1196032=655360} 2024-09-17T17:30:17.338 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.339 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.339 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:30:17.341 INFO:tasks.rados.rados.0.smithi102.stdout:229: writing smithi10238019-12 from 0 to 688128 tid 1 2024-09-17T17:30:17.344 INFO:tasks.rados.rados.0.smithi102.stdout:229: writing smithi10238019-12 from 688128 to 1196032 tid 2 2024-09-17T17:30:17.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:17.349 INFO:tasks.rados.rados.0.smithi102.stdout:229: writing smithi10238019-12 from 1196032 to 1851392 tid 3 2024-09-17T17:30:17.349 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 11 current snap is 17 2024-09-17T17:30:17.349 INFO:tasks.rados.rados.0.smithi102.stdout:230: seq_num 111 ranges {0=704512,704512=655360,1359872=507904} 2024-09-17T17:30:17.354 INFO:tasks.rados.rados.0.smithi102.stdout:230: writing smithi10238019-11 from 0 to 704512 tid 1 2024-09-17T17:30:17.358 INFO:tasks.rados.rados.0.smithi102.stdout:230: writing smithi10238019-11 from 704512 to 1359872 tid 2 2024-09-17T17:30:17.362 INFO:tasks.rados.rados.0.smithi102.stdout:230: writing smithi10238019-11 from 1359872 to 1867776 tid 3 2024-09-17T17:30:17.362 INFO:tasks.rados.rados.0.smithi102.stdout:231: setattr oid 47 current snap is 17 2024-09-17T17:30:17.362 INFO:tasks.rados.rados.0.smithi102.stdout:228: expect (ObjNum 93 snap 11 seq_num 93) 2024-09-17T17:30:17.362 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:17.382 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 361670 2024-09-17T17:30:17.395 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:17.399 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:17.397+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:17.413 INFO:tasks.rados.rados.0.smithi102.stdout:226: finishing copy_from racing read to smithi10238019-40 2024-09-17T17:30:17.413 INFO:tasks.rados.rados.0.smithi102.stdout:229: finishing write tid 1 to smithi10238019-12 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: finishing write tid 2 to smithi10238019-12 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:227: finishing copy_from racing read to smithi10238019-38 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:226: finishing copy_from to smithi10238019-40 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 40 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: finishing write tid 3 to smithi10238019-12 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: finishing write tid 4 to smithi10238019-12 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: finishing write tid 5 to smithi10238019-12 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: oid 12 updating version 0 to 85 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: oid 12 updating version 85 to 88 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: oid 12 version 88 is already newer than 86 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: oid 12 version 88 is already newer than 87 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 12 v 88 (ObjNum 110 snap 17 seq_num 110) dirty exists 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:229: left oid 12 (ObjNum 110 snap 17 seq_num 110) 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:230: finishing write tid 1 to smithi10238019-11 2024-09-17T17:30:17.414 INFO:tasks.rados.rados.0.smithi102.stdout:227: finishing copy_from to smithi10238019-38 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 38 v 21 (ObjNum 64 snap 3 seq_num 64) dirty exists 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: finishing write tid 2 to smithi10238019-11 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: finishing write tid 3 to smithi10238019-11 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: finishing write tid 4 to smithi10238019-11 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: finishing write tid 5 to smithi10238019-11 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: oid 11 updating version 0 to 86 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: oid 11 version 86 is already newer than 84 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: oid 11 version 86 is already newer than 83 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: oid 11 version 86 is already newer than 85 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 11 v 86 (ObjNum 111 snap 17 seq_num 111) dirty exists 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:230: left oid 11 (ObjNum 111 snap 17 seq_num 111) 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:226: done (5 left) 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:227: done (4 left) 2024-09-17T17:30:17.415 INFO:tasks.rados.rados.0.smithi102.stdout:228: done (3 left) 2024-09-17T17:30:17.416 INFO:tasks.rados.rados.0.smithi102.stdout:229: done (2 left) 2024-09-17T17:30:17.416 INFO:tasks.rados.rados.0.smithi102.stdout:230: done (1 left) 2024-09-17T17:30:17.416 INFO:tasks.rados.rados.0.smithi102.stdout:232: rollback oid 33 current snap is 17 2024-09-17T17:30:17.416 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 33 to 14 2024-09-17T17:30:17.416 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 34 current snap is 17 2024-09-17T17:30:17.416 INFO:tasks.rados.rados.0.smithi102.stdout:233: seq_num 112 ranges {0=720896,720896=802816,1523712=360448} 2024-09-17T17:30:17.417 INFO:tasks.rados.rados.0.smithi102.stdout:233: writing smithi10238019-34 from 0 to 720896 tid 1 2024-09-17T17:30:17.422 INFO:tasks.rados.rados.0.smithi102.stdout:233: writing smithi10238019-34 from 720896 to 1523712 tid 2 2024-09-17T17:30:17.424 INFO:tasks.rados.rados.0.smithi102.stdout:233: writing smithi10238019-34 from 1523712 to 1884160 tid 3 2024-09-17T17:30:17.424 INFO:tasks.rados.rados.0.smithi102.stdout:234: snap_create 2024-09-17T17:30:17.425 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 47 v 89 (ObjNum 65 snap 0 seq_num 3462598989) dirty exists 2024-09-17T17:30:17.425 INFO:tasks.rados.rados.0.smithi102.stdout:232: finishing rollback tid 0 to smithi10238019-33 2024-09-17T17:30:17.425 INFO:tasks.rados.rados.0.smithi102.stdout:232: finishing rollback tid 1 to smithi10238019-33 2024-09-17T17:30:17.425 INFO:tasks.rados.rados.0.smithi102.stdout:233: finishing write tid 1 to smithi10238019-34 2024-09-17T17:30:17.426 INFO:tasks.rados.rados.0.smithi102.stdout:233: finishing write tid 2 to smithi10238019-34 2024-09-17T17:30:17.427 INFO:tasks.rados.rados.0.smithi102.stdout:233: finishing write tid 3 to smithi10238019-34 2024-09-17T17:30:17.427 INFO:tasks.rados.rados.0.smithi102.stdout:233: finishing write tid 4 to smithi10238019-34 2024-09-17T17:30:17.428 INFO:tasks.rados.rados.0.smithi102.stdout:233: finishing write tid 5 to smithi10238019-34 2024-09-17T17:30:17.429 INFO:tasks.rados.rados.0.smithi102.stdout:233: oid 34 updating version 0 to 87 2024-09-17T17:30:17.429 INFO:tasks.rados.rados.0.smithi102.stdout:233: oid 34 updating version 87 to 89 2024-09-17T17:30:17.429 INFO:tasks.rados.rados.0.smithi102.stdout:233: oid 34 version 89 is already newer than 88 2024-09-17T17:30:17.429 INFO:tasks.rados.rados.0.smithi102.stdout:233: oid 34 updating version 89 to 90 2024-09-17T17:30:17.429 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 34 v 90 (ObjNum 112 snap 17 seq_num 112) dirty exists 2024-09-17T17:30:17.429 INFO:tasks.rados.rados.0.smithi102.stdout:233: left oid 34 (ObjNum 112 snap 17 seq_num 112) 2024-09-17T17:30:17.464 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.464 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.466 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:17.464+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:17.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:17.567 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:17.565+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:17.613 INFO:tasks.rados.rados.0.smithi102.stdout:232: finishing rollback tid 2 to smithi10238019-33 2024-09-17T17:30:17.613 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 33 v 23 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-17T17:30:17.620 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.620 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:17.668 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:17.666+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:17.736 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.736 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:17.769 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:17.768+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:17.852 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.852 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:17.870 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:17.869+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:17.968 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:17.968 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:17.971 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:17.969+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:17.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:18.072 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:18.070+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:18.083 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.083 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:18.173 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:18.171+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:18.212 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.212 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:18.274 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:18.272+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:18.330 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.330 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:18.374 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:18.372+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:18.431 INFO:tasks.rados.rados.0.smithi102.stdout:231: done (3 left) 2024-09-17T17:30:18.433 INFO:tasks.rados.rados.0.smithi102.stdout:232: done (2 left) 2024-09-17T17:30:18.433 INFO:tasks.rados.rados.0.smithi102.stdout:233: done (1 left) 2024-09-17T17:30:18.433 INFO:tasks.rados.rados.0.smithi102.stdout:234: done (0 left) 2024-09-17T17:30:18.433 INFO:tasks.rados.rados.0.smithi102.stdout:235: setattr oid 5 current snap is 18 2024-09-17T17:30:18.433 INFO:tasks.rados.rados.0.smithi102.stdout:236: snap_remove snap 16 2024-09-17T17:30:18.451 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.452 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:18.475 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:18.473+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:18.568 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.568 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.576 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:18.574+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:18.577 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:18.676 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:18.675+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:18.684 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.684 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:18.777 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:18.775+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:18.840 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.841 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.849 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:18.878 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:18.877+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:18.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:18.956 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:18.956 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:18.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:18.979 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:18.977+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:19.039 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:19.061 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:19.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:19.080 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:19.079+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:19.082 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:19.104 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:19.125 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:19.146 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:19.169 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:19.182 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:19.180+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:19.186 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:19.187 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:19.195 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:19.210 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:19.232 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:19.253 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:19.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:19.282 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:19.281+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:19.303 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:19.303 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:19.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:19.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:30:19.383 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:19.382+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:19.418 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:19.418 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:19.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:19.434 INFO:tasks.rados.rados.0.smithi102.stdout:236: done (1 left) 2024-09-17T17:30:19.434 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 46 current snap is 18 2024-09-17T17:30:19.435 INFO:tasks.rados.rados.0.smithi102.stdout:237: seq_num 113 ranges {3309568=737280,4046848=557056,4603904=606208} 2024-09-17T17:30:19.461 INFO:tasks.rados.rados.0.smithi102.stdout:237: writing smithi10238019-46 from 3309568 to 4046848 tid 1 2024-09-17T17:30:19.464 INFO:tasks.rados.rados.0.smithi102.stdout:237: writing smithi10238019-46 from 4046848 to 4603904 tid 2 2024-09-17T17:30:19.468 INFO:tasks.rados.rados.0.smithi102.stdout:237: writing smithi10238019-46 from 4603904 to 5210112 tid 3 2024-09-17T17:30:19.468 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 35 current snap is 18 2024-09-17T17:30:19.468 INFO:tasks.rados.rados.0.smithi102.stdout:238: seq_num 114 ranges {3571712=753664,4325376=688128,5013504=475136} 2024-09-17T17:30:19.483 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:19.482+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:19.492 INFO:tasks.rados.rados.0.smithi102.stdout:238: writing smithi10238019-35 from 3571712 to 4325376 tid 1 2024-09-17T17:30:19.496 INFO:tasks.rados.rados.0.smithi102.stdout:238: writing smithi10238019-35 from 4325376 to 5013504 tid 2 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:238: writing smithi10238019-35 from 5013504 to 5488640 tid 3 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:239: setattr oid 9 current snap is 18 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 5 v 46 (ObjNum 107 snap 16 seq_num 107) dirty exists 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:237: finishing write tid 1 to smithi10238019-46 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:237: finishing write tid 2 to smithi10238019-46 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:237: finishing write tid 3 to smithi10238019-46 2024-09-17T17:30:19.499 INFO:tasks.rados.rados.0.smithi102.stdout:238: finishing write tid 1 to smithi10238019-35 2024-09-17T17:30:19.500 INFO:tasks.rados.rados.0.smithi102.stdout:235: done (3 left) 2024-09-17T17:30:19.500 INFO:tasks.rados.rados.0.smithi102.stdout:240: setattr oid 50 current snap is 18 2024-09-17T17:30:19.501 INFO:tasks.rados.rados.0.smithi102.stdout:241: read oid 3 snap -1 2024-09-17T17:30:19.501 INFO:tasks.rados.rados.0.smithi102.stdout:241: expect (ObjNum 76 snap 4 seq_num 76) 2024-09-17T17:30:19.501 INFO:tasks.rados.rados.0.smithi102.stdout:242: read oid 2 snap -1 2024-09-17T17:30:19.501 INFO:tasks.rados.rados.0.smithi102.stdout:242: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-17T17:30:19.502 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 25 current snap is 18 2024-09-17T17:30:19.502 INFO:tasks.rados.rados.0.smithi102.stdout:243: seq_num 115 ranges {2064384=770048,2834432=704512,3538944=458752} 2024-09-17T17:30:19.503 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:19.517 INFO:tasks.rados.rados.0.smithi102.stdout:243: writing smithi10238019-25 from 2064384 to 2834432 tid 1 2024-09-17T17:30:19.521 INFO:tasks.rados.rados.0.smithi102.stdout:243: writing smithi10238019-25 from 2834432 to 3538944 tid 2 2024-09-17T17:30:19.524 INFO:tasks.rados.rados.0.smithi102.stdout:243: writing smithi10238019-25 from 3538944 to 3997696 tid 3 2024-09-17T17:30:19.524 INFO:tasks.rados.rados.0.smithi102.stdout:244: delete oid 18 current snap is 18 2024-09-17T17:30:19.524 INFO:tasks.rados.rados.0.smithi102.stdout:238: finishing write tid 2 to smithi10238019-35 2024-09-17T17:30:19.525 INFO:tasks.rados.rados.0.smithi102.stdout:238: finishing write tid 3 to smithi10238019-35 2024-09-17T17:30:19.525 INFO:tasks.rados.rados.0.smithi102.stdout:242: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-17T17:30:19.525 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 192631 2024-09-17T17:30:19.526 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:19.530+0000 7f6b0da88640 0 -- 172.21.15.7:0/2818200500 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f6b0821f280 msgr2=0x7f6b08221670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:19.541 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:19.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:19.554 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:19.554 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:19.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:19.554 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:308 3.2s0 3:4c833192:::smithi10238019-39:head [rollback 5] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.581499087000001, 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076693619999999999, 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 308 2024-09-17T17:30:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.68e-07 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970369+0000", 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7160000000000002e-06 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970378+0000", 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5739999999999996e-06 2024-09-17T17:30:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970383+0000", 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0010000000000001e-06 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971568+0000", 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011857020000000001 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971627+0000", 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8826999999999998e-05 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972103+0000", 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047574100000000001 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973681+0000", 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001578404 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.977986+0000", 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043046630000000002 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978033+0000", 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7166000000000002e-05 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:309 3.2s0 3:4c833192:::smithi10238019-39:head [append 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.581468045000001, 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077796990000000002, 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 309 2024-09-17T17:30:19.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2e-07 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970399+0000", 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0779999999999997e-06 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970408+0000", 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7979999999999995e-06 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970415+0000", 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9310000000000002e-06 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972569+0000", 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021534969999999999 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972677+0000", 2024-09-17T17:30:19.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108145 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973250+0000", 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057328100000000003 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973923+0000", 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00067244899999999999 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978120+0000", 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041977949999999998 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978175+0000", 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4205e-05 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:326 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698785+0000", 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.853078373999999, 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068833499999999999, 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 326 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698785+0000", 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698785+0000", 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698787+0000", 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7209999999999999e-06 2024-09-17T17:30:19.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698798+0000", 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1032999999999999e-05 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698819+0000", 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1005999999999998e-05 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698830+0000", 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0858e-05 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698902+0000", 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1996000000000007e-05 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698911+0000", 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1859999999999998e-06 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.701053+0000", 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002141479 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.701152+0000", 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9332000000000004e-05 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702018+0000", 2024-09-17T17:30:19.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086619400000000004 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704464+0000", 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024453949999999999 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.705564+0000", 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001100668 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.705668+0000", 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103482 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:327 3.2s0 3:497089da:::smithi10238019-9:head [rollback 5] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.853013623999999, 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0072194929999999996, 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 327 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698850+0000", 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698852+0000", 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3939999999999999e-06 2024-09-17T17:30:19.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698858+0000", 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1480000000000003e-06 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698863+0000", 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3470000000000001e-06 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698988+0000", 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012491999999999999 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698997+0000", 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6819999999999998e-06 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702506+0000", 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035088879999999999 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702618+0000", 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111428 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703259+0000", 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064172000000000003 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704549+0000", 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001289599 2024-09-17T17:30:19.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706021+0000", 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001471857 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706069+0000", 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8072000000000003e-05 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:328 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.852974081999999, 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0073597310000000004, 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 328 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.580 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7e-07 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698894+0000", 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.057e-06 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698902+0000", 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3760000000000002e-06 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698907+0000", 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0329999999999999e-06 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699035+0000", 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012849700000000001 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699042+0000", 2024-09-17T17:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.55e-06 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703631+0000", 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004589536 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703735+0000", 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103891 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704152+0000", 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041683699999999998 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704821+0000", 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066855199999999999 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706206+0000", 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001385788 2024-09-17T17:30:19.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706249+0000", 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2243999999999997e-05 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 52/35 ECSubWrite(tid=125, reqid=osd.13.0:125, at_version=52'57, trim_to=0'0, roll_forward_to=51'53))", 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.207782119000001, 2024-09-17T17:30:19.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0071485719999999997, 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344082+0000", 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.85e-07 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344108+0000", 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6270000000000001e-05 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344147+0000", 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9462999999999998e-05 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344158+0000", 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0786e-05 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344212+0000", 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4143e-05 2024-09-17T17:30:19.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344220+0000", 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8390000000000007e-06 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.351178+0000", 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006957876 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.351229+0000", 2024-09-17T17:30:19.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1409999999999997e-05 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:421 3.2s0 3:4c833192:::smithi10238019-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.169221858, 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090389679999999997, 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:19.591 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 421 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:19.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382642+0000", 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000005e-07 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382644+0000", 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382652+0000", 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5059999999999998e-06 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382656+0000", 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8029999999999996e-06 2024-09-17T17:30:19.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385875+0000", 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0032189900000000001 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385954+0000", 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9087999999999999e-05 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388823+0000", 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028687090000000001 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388909+0000", 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5457000000000005e-05 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.391680+0000", 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027713939999999999 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.596 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 53/35 ECSubWrite(tid=133, reqid=osd.13.0:133, at_version=53'64, trim_to=0'0, roll_forward_to=52'62))", 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.189818673, 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078716470000000007, 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:30:19.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362068+0000", 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3370000000000002e-05 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362467+0000", 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039876899999999999 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362476+0000", 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8740000000000001e-06 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364044+0000", 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001568261 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364055+0000", 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.077e-05 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369861+0000", 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0058066660000000003 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369916+0000", 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4691999999999997e-05 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 53/35 ECSubWrite(tid=137, reqid=client.4566.0:461, at_version=53'70, trim_to=0'0, roll_forward_to=53'66))", 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:04.377174+0000", 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.174688602, 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067048919999999996, 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377174+0000", 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377174+0000", 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377175+0000", 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.031e-06 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377603+0000", 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00042773600000000001 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377639+0000", 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6149999999999998e-05 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377648+0000", 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0759999999999992e-06 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377714+0000", 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6014999999999997e-05 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.377725+0000", 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0249e-05 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.383847+0000", 2024-09-17T17:30:19.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0061220110000000001 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.383879+0000", 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2623999999999998e-05 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s5 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.212445534, 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074908880000000002, 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339418+0000", 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339449+0000", 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.133e-05 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339554+0000", 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104682 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339563+0000", 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9129999999999992e-06 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339937+0000", 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037396800000000002 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339947+0000", 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0451999999999999e-05 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346872+0000", 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069250020000000004 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346908+0000", 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6117e-05 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs3 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.339535+0000", 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.212328266, 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068455240000000004, 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339535+0000", 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339535+0000", 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339536+0000", 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0800000000000003e-07 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339563+0000", 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6982999999999999e-05 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339600+0000", 2024-09-17T17:30:19.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7599999999999999e-05 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339608+0000", 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1499999999999999e-06 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339660+0000", 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1758999999999997e-05 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339680+0000", 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0173000000000002e-05 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346338+0000", 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006658181 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346380+0000", 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1770000000000002e-05 2024-09-17T17:30:19.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s2 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.378147+0000", 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.173715731, 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067666209999999996, 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378147+0000", 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378147+0000", 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378148+0000", 2024-09-17T17:30:19.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.279e-06 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378184+0000", 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5629000000000001e-05 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378240+0000", 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5699999999999999e-05 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378249+0000", 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.054e-06 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378313+0000", 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4425999999999996e-05 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378321+0000", 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6490000000000008e-06 2024-09-17T17:30:19.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.384869+0000", 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0065477260000000002 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.384914+0000", 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5157999999999997e-05 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=154, reqid=osd.13.0:154, at_version=57'84, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.161948062, 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035993311, 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389916+0000", 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0299999999999997e-07 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389936+0000", 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0061e-05 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425016+0000", 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035080622999999998 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425025+0000", 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3110000000000006e-06 2024-09-17T17:30:19.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425088+0000", 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3442999999999995e-05 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425103+0000", 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4625e-05 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425875+0000", 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000772585 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425908+0000", 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2759999999999998e-05 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=155, reqid=osd.13.0:155, at_version=57'86, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:30:19.625 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.161046882999999, 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26589276499999998, 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390817+0000", 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390839+0000", 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2725999999999999e-05 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655507+0000", 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26466748600000001 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655515+0000", 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5530000000000004e-06 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655625+0000", 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109997 2024-09-17T17:30:19.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655635+0000", 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0000000000000001e-05 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656673+0000", 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001038464 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656709+0000", 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5830000000000001e-05 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=156, reqid=client.4566.0:492, at_version=57'10, trim_to=0'0, roll_forward_to=43'8))", 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.126886408000001, 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23215693400000001, 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4900000000000004e-07 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424998+0000", 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0800000000000001e-05 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655688+0000", 2024-09-17T17:30:19.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23068955499999999 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655696+0000", 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0839999999999992e-06 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655818+0000", 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012232399999999999 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655830+0000", 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1355000000000001e-05 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657105+0000", 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001275403 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657133+0000", 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8464000000000001e-05 2024-09-17T17:30:19.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=158, reqid=client.4566.0:492, at_version=0'0, trim_to=0'0, roll_forward_to=57'10))", 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.894041345, 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26690073399999997, 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000001e-07 2024-09-17T17:30:19.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657834+0000", 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1450999999999999e-05 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923785+0000", 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26595101999999998 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923800+0000", 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.508e-05 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923956+0000", 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155876 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923975+0000", 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9443e-05 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924688+0000", 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071237099999999995 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924722+0000", 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4629000000000003e-05 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 59/35 ECSubWrite(tid=164, reqid=client.4566.0:513, at_version=59'90, trim_to=0'0, roll_forward_to=59'88))", 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.1763206670000006, 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080861479999999996, 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375543+0000", 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4499999999999995e-07 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375918+0000", 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037515399999999999 2024-09-17T17:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375961+0000", 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2206000000000002e-05 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375969+0000", 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8349999999999993e-06 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376050+0000", 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0269e-05 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376060+0000", 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0509999999999999e-05 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383584+0000", 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075235939999999998 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383628+0000", 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4635000000000002e-05 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 59/35 ECSubWrite(tid=165, reqid=client.4566.0:514, at_version=59'91, trim_to=0'0, roll_forward_to=59'88))", 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:10.376940+0000", 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.1749229400000001, 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067160639999999999, 2024-09-17T17:30:19.644 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376940+0000", 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376940+0000", 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376941+0000", 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376966+0000", 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5289000000000001e-05 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376998+0000", 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1542999999999999e-05 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.377004+0000", 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3439999999999997e-06 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.377075+0000", 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0889000000000005e-05 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.377085+0000", 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0509999999999999e-05 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383635+0000", 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0065491600000000001 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383656+0000", 2024-09-17T17:30:19.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1535e-05 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:562 3.2s0 3:497089da:::smithi10238019-9:head [rollback f] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.1357293359999998, 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076874359999999997, 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 562 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1299999999999999e-07 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416138+0000", 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4920000000000002e-06 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416146+0000", 2024-09-17T17:30:19.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8800000000000008e-06 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416150+0000", 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6840000000000004e-06 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416271+0000", 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012026799999999999 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416278+0000", 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5889999999999997e-06 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419622+0000", 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0033434419999999999 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419740+0000", 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011798600000000001 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420292+0000", 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00055223900000000005 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421629+0000", 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001337429 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423776+0000", 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021470370000000001 2024-09-17T17:30:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423821+0000", 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4676999999999999e-05 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:563 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.1357054839999998, 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079319400000000002, 2024-09-17T17:30:19.655 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 563 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416158+0000", 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416161+0000", 2024-09-17T17:30:19.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6649999999999999e-06 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416171+0000", 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0176e-05 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416181+0000", 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0327e-05 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416310+0000", 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129247 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416317+0000", 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0720000000000002e-06 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420645+0000", 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043279629999999998 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420731+0000", 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5937000000000001e-05 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421175+0000", 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044333000000000003 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423834+0000", 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002659121 2024-09-17T17:30:19.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424046+0000", 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021196999999999999 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424089+0000", 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3702999999999998e-05 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:19.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:19.662 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:19.663 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:19.664 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1617539 2024-09-17T17:30:19.664 INFO:tasks.rados.rados.0.smithi102.stdout:241: expect (ObjNum 76 snap 4 seq_num 76) 2024-09-17T17:30:19.664 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:19.664 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 954514 2024-09-17T17:30:19.664 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 2600511 2024-09-17T17:30:19.664 INFO:tasks.rados.rados.0.smithi102.stdout:238: finishing write tid 4 to smithi10238019-35 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:238: finishing write tid 5 to smithi10238019-35 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:238: oid 35 updating version 0 to 94 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:238: oid 35 updating version 94 to 97 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:238: oid 35 version 97 is already newer than 96 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:238: oid 35 version 97 is already newer than 95 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 35 v 97 (ObjNum 114 snap 18 seq_num 114) dirty exists 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:238: left oid 35 (ObjNum 114 snap 18 seq_num 114) 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:243: finishing write tid 1 to smithi10238019-25 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:243: finishing write tid 2 to smithi10238019-25 2024-09-17T17:30:19.665 INFO:tasks.rados.rados.0.smithi102.stdout:243: finishing write tid 3 to smithi10238019-25 2024-09-17T17:30:19.666 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:19.666 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:19.661+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:19.690 INFO:tasks.rados.rados.0.smithi102.stdout:238: done (7 left) 2024-09-17T17:30:19.690 INFO:tasks.rados.rados.0.smithi102.stdout:241: done (6 left) 2024-09-17T17:30:19.691 INFO:tasks.rados.rados.0.smithi102.stdout:242: done (5 left) 2024-09-17T17:30:19.691 INFO:tasks.rados.rados.0.smithi102.stdout:244: done (4 left) 2024-09-17T17:30:19.691 INFO:tasks.rados.rados.0.smithi102.stdout:245: rmattr oid 8 current snap is 18 2024-09-17T17:30:19.691 INFO:tasks.rados.rados.0.smithi102.stdout:246: delete oid 10 current snap is 18 2024-09-17T17:30:19.691 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 8 v 0 (ObjNum 99 snap 13 seq_num 99) dirty exists 2024-09-17T17:30:19.693 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 50 v 121 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-17T17:30:19.699 INFO:tasks.rados.rados.0.smithi102.stdout:240: done (5 left) 2024-09-17T17:30:19.699 INFO:tasks.rados.rados.0.smithi102.stdout:245: done (4 left) 2024-09-17T17:30:19.700 INFO:tasks.rados.rados.0.smithi102.stdout:246: done (3 left) 2024-09-17T17:30:19.700 INFO:tasks.rados.rados.0.smithi102.stdout:247: delete oid 3 current snap is 18 2024-09-17T17:30:19.764 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:19.763+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:19.770 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:19.771 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:19.771 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:19.771 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:19.779 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:19.865 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:19.863+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:19.925 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:19.926 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:19.926 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:19.926 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:19.926 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:19.934 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:19.974 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:19.973+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:20.066 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:20.064+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:20.077 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:169 3.as0 3:5ee91470:::smithi10238019-6:head [read 83682~14622,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 31.084918571999999, 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67448544799999999, 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.078 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 169 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:20.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990278+0000", 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6400000000000002e-07 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990283+0000", 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3900000000000001e-06 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990291+0000", 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9379999999999992e-06 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990297+0000", 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6099999999999997e-06 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992308+0000", 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0020111650000000001 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992380+0000", 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1459999999999997e-05 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.664763+0000", 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67238332099999998 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 37/32 ECSubWriteReply(tid=24, last_complete=37'12, committed=1, applied=1))", 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 31.074886729999999, 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069166769000000003, 2024-09-17T17:30:20.083 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:20.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000310+0000", 2024-09-17T17:30:20.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1299999999999998e-07 2024-09-17T17:30:20.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000314+0000", 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.461e-06 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069373+0000", 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069059000999999995 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069380+0000", 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2570000000000001e-06 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069437+0000", 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7519999999999998e-05 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069476+0000", 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8617e-05 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.090 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 31.005852414, 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52672497699999998, 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069345+0000", 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.048e-06 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069355+0000", 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0052e-05 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595186+0000", 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52583139400000001 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595201+0000", 2024-09-17T17:30:20.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4790000000000001e-05 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595348+0000", 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014743699999999999 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595363+0000", 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4589999999999999e-05 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596033+0000", 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067039499999999995 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596069+0000", 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5271000000000002e-05 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:222 3.as0 3:5a070e56:::smithi10238019-47:head [read 191490~594942,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.725970+0000", 2024-09-17T17:30:20.095 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.349226388000002, 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079859110000000001, 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 222 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.725970+0000", 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.725970+0000", 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.725970+0000", 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1699999999999998e-07 2024-09-17T17:30:20.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.725974+0000", 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0069999999999999e-06 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.725984+0000", 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3249999999999997e-06 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.725990+0000", 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0079999999999999e-06 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726515+0000", 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00052583700000000003 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726606+0000", 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0067000000000004e-05 2024-09-17T17:30:20.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.733956+0000", 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073500500000000003 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:223 3.as0 3:5a070e56:::smithi10238019-47:head [read 351913~434519,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.349183123, 2024-09-17T17:30:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010342224000000001, 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 223 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:20.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7299999999999996e-07 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726018+0000", 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0770000000000001e-06 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726027+0000", 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7210000000000006e-06 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726034+0000", 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3419999999999996e-06 2024-09-17T17:30:20.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.728268+0000", 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002233886 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.728356+0000", 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8360000000000001e-05 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.736355+0000", 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079992650000000002 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:303 3.as0 3:5ee91470:::smithi10238019-6:head [append 0~491520] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.965533+0000", 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.109663374, 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078317379999999995, 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.105 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 303 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965533+0000", 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965533+0000", 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965534+0000", 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.457e-06 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965863+0000", 2024-09-17T17:30:20.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00032853899999999998 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965876+0000", 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2899e-05 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965882+0000", 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1879999999999999e-06 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965929+0000", 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7698999999999999e-05 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.965998+0000", 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8255000000000003e-05 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.966645+0000", 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00064766900000000002 2024-09-17T17:30:20.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.968139+0000", 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001493147 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.973280+0000", 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0051413479999999996 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.973364+0000", 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4536999999999994e-05 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:304 3.as0 3:5ee91470:::smithi10238019-6:head [append 0~770048] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970218+0000", 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.104978215999999, 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078065440000000003, 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.111 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 304 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970218+0000", 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970218+0000", 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970219+0000", 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.051e-06 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970622+0000", 2024-09-17T17:30:20.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00040295199999999999 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970633+0000", 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1435e-05 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970640+0000", 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1679999999999998e-06 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970693+0000", 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2389000000000003e-05 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970752+0000", 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8847000000000001e-05 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.972007+0000", 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0012549569999999999 2024-09-17T17:30:20.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977686+0000", 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0056791029999999996 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977946+0000", 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000260504 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978024+0000", 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8138000000000003e-05 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:306 3.as0 3:5ee91470:::smithi10238019-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970702+0000", 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.104493743999999, 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090032800000000007, 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.117 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 306 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970702+0000", 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970702+0000", 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970703+0000", 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.75e-07 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970706+0000", 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9749999999999999e-06 2024-09-17T17:30:20.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970713+0000", 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4030000000000001e-06 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970719+0000", 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5969999999999999e-06 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.974664+0000", 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0039454750000000004 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.974755+0000", 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0650999999999997e-05 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978620+0000", 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0038654449999999999 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978710+0000", 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9321999999999996e-05 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.979706+0000", 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00099583700000000007 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:384 3.as0 3:5673cc6d:::smithi10238019-2:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.353290+0000", 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.721906490999999, 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078850670000000008, 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 384 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.123 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353290+0000", 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353290+0000", 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353290+0000", 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3700000000000001e-07 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353294+0000", 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0219999999999998e-06 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353301+0000", 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3590000000000002e-06 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353306+0000", 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.651e-06 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.358146+0000", 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048403099999999996 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.358222+0000", 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5947999999999999e-05 2024-09-17T17:30:20.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.361175+0000", 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0029523399999999999 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:385 3.as0 3:5673cc6d:::smithi10238019-2:head [read 1766~31002,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.353313+0000", 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.721883163000001, 2024-09-17T17:30:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0084466260000000005, 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 385 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353313+0000", 2024-09-17T17:30:20.128 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353313+0000", 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353313+0000", 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4200000000000001e-07 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353316+0000", 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5239999999999999e-06 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353323+0000", 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1509999999999998e-06 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.353329+0000", 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1229999999999997e-06 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.358472+0000", 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0051428250000000002 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.358535+0000", 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3303e-05 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.361760+0000", 2024-09-17T17:30:20.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0032242579999999998 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:390 3.as0 3:5ee91470:::smithi10238019-6:head [create,setxattr `sjUtrS\\bejkdS^PZ^UbhfeR]sl^p__anSUR_aom (1578) in=40b,setxattr Xgg (981) in=3b,setxattr jjU_`S[ao\\\\^aUS`bZo^cefhdeaegSZfc]sS[m[k (631) in=40b,setxattr WptVXs^]ZljihqWVjRTTRsYbpoOd[kede^[d^iNS (15648) in=40b,setxattr NUPa_VdcRWiTY (981) in=13b,setxattr oapqrhbaXndiek^ec[NZPnlNjeX^`^^ht`__ll[g (630) in=40b,setxattr hUokalb_dNqZaP[SZdSptRb]ZndqQTkPSdWpXlg_ (1683) in=40b,setxattr SOaUaOtoOmejUfsqfadVc_\\fpWak\\^nknRQfaXOZ (9120) in=40b,setxattr r[RO_Pl (981) in=7b,setxattr OOd^ZXYi[]NO^mlSkQafkPlVaY`UYcd\\bnaQUZjq (1225) in=40b,setxattr mlY]_qqgYNhaYetZtkgQPN^a^VmW[amS`Qbadgjr (137) in=40b,setxattr iVYdjchT[j\\[\\\\Uto`qnjUcXas^PtqZa^VoRktTT (1823) in=40b,setxattr PhV`]ZkggalnUas (981) in=15b,setxattr [TNbUtYOkT`ZatPerOQOlk\\WahmgadNWaeRZ^YT\\ (1121) in=40b,setxattr aRqq]f]b_cobaZ`ZZRaattjja[X``PaT\\l\\^Y[jU (313) in=40b,setxattr qYXjVRWg (981) in=8b,setxattr cm`nfp^lmRcSZUsStaOg_omiONqeTRWUsOjm]png (1227) in=40b,setxattr _jtbmRhQeZe]nY\\TQctPVagWm[isibp]ooW`hq[Z (1367) in=40b,setxattr _Ybhal\\]RPkmpUPPRRb[_[YfQpj[aXsgkfNbeeOZ (946) in=40b,setxattr YeQo_aSrmVSirmpTkkZqfZaaqOtg\\X\\qoRPrjPai (1786) in=40b] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.742779901999999, 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096219809999999999, 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.132 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 390 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:20.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332418+0000", 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3379999999999999e-06 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332463+0000", 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4666999999999998e-05 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.134 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332482+0000", 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8822e-05 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332493+0000", 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0686000000000001e-05 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332566+0000", 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3250999999999994e-05 2024-09-17T17:30:20.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332571+0000", 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1479999999999997e-06 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.335092+0000", 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0025209799999999999 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.335166+0000", 2024-09-17T17:30:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3672999999999997e-05 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.336267+0000", 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011013399999999999 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.338033+0000", 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001766323 2024-09-17T17:30:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.341931+0000", 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0038977349999999998 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342038+0000", 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107018 2024-09-17T17:30:20.138 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:395 3.as0 3:5280ab18:::smithi10238019-18:head [append 0~622592] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.342310+0000", 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.732885983999999, 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090640670000000003, 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.139 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 395 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342310+0000", 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342310+0000", 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342312+0000", 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1780000000000002e-06 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342989+0000", 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067656599999999997 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343006+0000", 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7578999999999999e-05 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343015+0000", 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3629999999999994e-06 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343065+0000", 2024-09-17T17:30:20.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9790000000000003e-05 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343157+0000", 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2149000000000002e-05 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.344029+0000", 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00087232900000000003 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.348159+0000", 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041304050000000002 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.351310+0000", 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0031507710000000001 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.351374+0000", 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3936999999999994e-05 2024-09-17T17:30:20.144 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:396 3.as0 3:5280ab18:::smithi10238019-18:head [append 0~147456] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.732006556999998, 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012450536999999999, 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.145 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 396 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:20.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343191+0000", 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0839999999999999e-06 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343250+0000", 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9113999999999997e-05 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343262+0000", 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2649999999999999e-05 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343270+0000", 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0879999999999995e-06 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.345733+0000", 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024625039999999999 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.345807+0000", 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3769000000000005e-05 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346281+0000", 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047393899999999999 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.350852+0000", 2024-09-17T17:30:20.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0045715920000000002 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355563+0000", 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047105319999999999 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355640+0000", 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.7265e-05 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:397 3.as0 3:5280ab18:::smithi10238019-18:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.73190628, 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012559297000000001, 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.151 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 397 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3399999999999999e-07 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343293+0000", 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1700000000000001e-06 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343299+0000", 2024-09-17T17:30:20.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3199999999999999e-06 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343302+0000", 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.495e-06 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346935+0000", 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0036331419999999998 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346988+0000", 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2828999999999999e-05 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347263+0000", 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000274999 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.350964+0000", 2024-09-17T17:30:20.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003701101 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355764+0000", 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047997869999999998 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355849+0000", 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4919999999999993e-05 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.156 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:398 3.as0 3:5280ab18:::smithi10238019-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.731888193, 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013718741, 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.157 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 398 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9700000000000003e-07 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343310+0000", 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.525e-06 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343314+0000", 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8770000000000003e-06 2024-09-17T17:30:20.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343316+0000", 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8080000000000001e-06 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347523+0000", 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042062560000000002 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347583+0000", 2024-09-17T17:30:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0201999999999997e-05 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356234+0000", 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086509659999999995 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356431+0000", 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00019758699999999999 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.357027+0000", 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00059522299999999998 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s2 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352290+0000", 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.722906031000001, 2024-09-17T17:30:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0080238080000000003, 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352290+0000", 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352290+0000", 2024-09-17T17:30:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352290+0000", 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4400000000000001e-07 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352314+0000", 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3991000000000001e-05 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352387+0000", 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2627999999999996e-05 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352395+0000", 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9419999999999995e-06 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352916+0000", 2024-09-17T17:30:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00052137400000000001 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352933+0000", 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6663e-05 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.360272+0000", 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073388330000000003 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.360314+0000", 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2132999999999998e-05 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 51/32 ECSubWriteReply(tid=64, last_complete=51'43, committed=1, applied=1))", 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.718421684999999, 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.78010271600000003, 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356775+0000", 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5000000000000001e-07 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356781+0000", 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.716e-06 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136139+0000", 2024-09-17T17:30:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.77935822499999996 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136153+0000", 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.359e-05 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136325+0000", 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017257700000000001 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136877+0000", 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00055165800000000001 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.8s6 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.337456+0000", 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.737739683000001, 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0093714709999999993, 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337456+0000", 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337456+0000", 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.172 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337457+0000", 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.92e-07 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337477+0000", 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0305999999999998e-05 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337511+0000", 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4422999999999999e-05 2024-09-17T17:30:20.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337519+0000", 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0369999999999995e-06 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337574+0000", 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5140999999999999e-05 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.174 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337585+0000", 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0743e-05 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.346797+0000", 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0092117180000000007 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.346828+0000", 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0911000000000003e-05 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.175 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s2 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.737267194999999, 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012584702999999999, 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.176 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.177 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337956+0000", 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6574e-05 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338017+0000", 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1050999999999995e-05 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338024+0000", 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2300000000000002e-06 2024-09-17T17:30:20.178 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338194+0000", 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017029299999999999 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338202+0000", 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.892e-06 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.179 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.350495+0000", 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012292684 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.350514+0000", 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8678999999999999e-05 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:641 3.as0 3:5280ab18:::smithi10238019-18:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:30:20.180 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.55149807500000003, 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.16562932699999999, 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 641 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.181 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.182 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523700+0000", 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1150000000000001e-06 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523708+0000", 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5599999999999996e-06 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523725+0000", 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7725999999999999e-05 2024-09-17T17:30:20.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523733+0000", 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5299999999999999e-06 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6327000000000004e-05 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.184 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523914+0000", 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114837 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524640+0000", 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072579000000000003 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526382+0000", 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0017423060000000001 2024-09-17T17:30:20.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689252+0000", 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.162869823 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689327+0000", 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5313000000000002e-05 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.186 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.187 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:20.188 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:20.189 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:20.187+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:20.194 INFO:tasks.rados.rados.0.smithi102.stdout:243: finishing write tid 4 to smithi10238019-25 2024-09-17T17:30:20.195 INFO:tasks.rados.rados.0.smithi102.stdout:243: finishing write tid 5 to smithi10238019-25 2024-09-17T17:30:20.195 INFO:tasks.rados.rados.0.smithi102.stdout:243: oid 25 updating version 0 to 101 2024-09-17T17:30:20.195 INFO:tasks.rados.rados.0.smithi102.stdout:243: oid 25 version 101 is already newer than 100 2024-09-17T17:30:20.195 INFO:tasks.rados.rados.0.smithi102.stdout:243: oid 25 updating version 101 to 102 2024-09-17T17:30:20.195 INFO:tasks.rados.rados.0.smithi102.stdout:243: oid 25 version 102 is already newer than 99 2024-09-17T17:30:20.196 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 25 v 102 (ObjNum 115 snap 18 seq_num 115) dirty exists 2024-09-17T17:30:20.196 INFO:tasks.rados.rados.0.smithi102.stdout:243: left oid 25 (ObjNum 115 snap 18 seq_num 115) 2024-09-17T17:30:20.225 INFO:tasks.rados.rados.0.smithi102.stdout:243: done (3 left) 2024-09-17T17:30:20.225 INFO:tasks.rados.rados.0.smithi102.stdout:247: done (2 left) 2024-09-17T17:30:20.225 INFO:tasks.rados.rados.0.smithi102.stdout:248: snap_create 2024-09-17T17:30:20.288 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:20.287+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:20.292 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:20.292 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:20.292 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:20.292 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:20.300 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:30:20.335 INFO:tasks.rados.rados.0.smithi102.stdout:248: done (2 left) 2024-09-17T17:30:20.335 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 36 current snap is 19 2024-09-17T17:30:20.336 INFO:tasks.rados.rados.0.smithi102.stdout:249: seq_num 119 ranges {0=458752,458752=507904,966656=507904,1474560=458752,1933312=81920} 2024-09-17T17:30:20.338 INFO:tasks.rados.rados.0.smithi102.stdout:249: writing smithi10238019-36 from 0 to 458752 tid 1 2024-09-17T17:30:20.341 INFO:tasks.rados.rados.0.smithi102.stdout:249: writing smithi10238019-36 from 458752 to 966656 tid 2 2024-09-17T17:30:20.344 INFO:tasks.rados.rados.0.smithi102.stdout:249: writing smithi10238019-36 from 966656 to 1474560 tid 3 2024-09-17T17:30:20.347 INFO:tasks.rados.rados.0.smithi102.stdout:249: writing smithi10238019-36 from 1474560 to 1933312 tid 4 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:249: writing smithi10238019-36 from 1933312 to 2015232 tid 5 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:250: read oid 2 snap -1 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:250: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:251: rmattr oid 48 current snap is 19 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:251: done (4 left) 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:252: rollback oid 31 current snap is 19 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 31 to 13 2024-09-17T17:30:20.348 INFO:tasks.rados.rados.0.smithi102.stdout:253: read oid 10 snap -1 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:253: expect deleted 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:254: copy_from oid 47 from oid 12 current snap is 19 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:255: delete oid 6 current snap is 19 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: finishing write tid 4 to smithi10238019-46 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: finishing write tid 5 to smithi10238019-46 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: oid 46 updating version 0 to 51 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: oid 46 version 51 is already newer than 49 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: oid 46 version 51 is already newer than 48 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: oid 46 version 51 is already newer than 50 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 51 (ObjNum 113 snap 18 seq_num 113) dirty exists 2024-09-17T17:30:20.349 INFO:tasks.rados.rados.0.smithi102.stdout:237: left oid 46 (ObjNum 113 snap 18 seq_num 113) 2024-09-17T17:30:20.350 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 44 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-17T17:30:20.352 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 1 to smithi10238019-36 2024-09-17T17:30:20.353 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 2 to smithi10238019-36 2024-09-17T17:30:20.353 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 3 to smithi10238019-36 2024-09-17T17:30:20.354 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 4 to smithi10238019-36 2024-09-17T17:30:20.356 INFO:tasks.rados.rados.0.smithi102.stdout:252: finishing rollback tid 0 to smithi10238019-31 2024-09-17T17:30:20.357 INFO:tasks.rados.rados.0.smithi102.stdout:252: finishing rollback tid 1 to smithi10238019-31 2024-09-17T17:30:20.357 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 43 (ObjNum 81 snap 0 seq_num 0) dirty dne 2024-09-17T17:30:20.357 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 5 to smithi10238019-36 2024-09-17T17:30:20.359 INFO:tasks.rados.rados.0.smithi102.stdout:250: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-17T17:30:20.370 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1792959 2024-09-17T17:30:20.371 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:20.389 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:20.396 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:20.394+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:254: finishing copy_from racing read to smithi10238019-47 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:254: finishing copy_from to smithi10238019-47 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 47 v 88 (ObjNum 110 snap 17 seq_num 110) dirty exists 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:237: done (7 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:239: done (6 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:250: done (5 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:252: done (4 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:253: done (3 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:254: done (2 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:255: done (1 left) 2024-09-17T17:30:20.410 INFO:tasks.rados.rados.0.smithi102.stdout:256: snap_remove snap 13 2024-09-17T17:30:20.448 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:20.448 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:20.448 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:20.449 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:20.449 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:20.456 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:20.498 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:20.496+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:20.591 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:20.590+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:20.602 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:20.602 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:20.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.bs5 43/32 ECSubWrite(tid=92, reqid=osd.13.0:92, at_version=43'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.697941+0000", 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.902112341999999, 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075441680000000004, 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697941+0000", 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697941+0000", 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697942+0000", 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4900000000000005e-07 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697962+0000", 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0424e-05 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697997+0000", 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4164e-05 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698005+0000", 2024-09-17T17:30:20.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3950000000000001e-06 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698083+0000", 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8208999999999996e-05 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698094+0000", 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0533000000000001e-05 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.705450+0000", 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073559539999999996 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.705485+0000", 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.574e-05 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:311 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 431326~1305378,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:20.607 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.629640389999999, 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085766909999999991, 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 311 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970414+0000", 2024-09-17T17:30:20.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.8299999999999997e-07 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970418+0000", 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9500000000000003e-06 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970425+0000", 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.452e-06 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970429+0000", 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4240000000000002e-06 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.971012+0000", 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00058375099999999997 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.971104+0000", 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1195999999999999e-05 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.611 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978990+0000", 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078864350000000007 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:312 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 0~1736704,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.629611655000001, 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50817852500000005, 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.612 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 312 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1600000000000002e-07 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970445+0000", 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7999999999999999e-06 2024-09-17T17:30:20.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970451+0000", 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0970000000000002e-06 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970456+0000", 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6009999999999997e-06 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976257+0000", 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005800889 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976348+0000", 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1614999999999996e-05 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478621+0000", 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50227210700000002 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.616 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.ds0 43/32 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.972479+0000", 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.627575078, 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0065331599999999997, 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.617 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.972479+0000", 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.972479+0000", 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.972479+0000", 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9699999999999996e-07 2024-09-17T17:30:20.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.973029+0000", 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054954600000000004 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978031+0000", 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0050022410000000002 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978037+0000", 2024-09-17T17:30:20.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5939999999999999e-06 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978086+0000", 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.8263999999999997e-05 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.979012+0000", 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00092601800000000002 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.620 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.ds0 43/32 ECSubReadReply(tid=23, attrs_read=0))", 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.623000962999999, 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.501606106, 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:20.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977054+0000", 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.16e-07 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978014+0000", 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096069199999999997 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477737+0000", 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.49972330100000001 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477745+0000", 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2130000000000002e-06 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477802+0000", 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7555e-05 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478659+0000", 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00085652899999999997 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=31, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 20.242182831000001, 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53920807500000001, 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357876+0000", 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.092e-06 2024-09-17T17:30:20.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896536+0000", 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53865993499999998 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896547+0000", 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1608e-05 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896653+0000", 2024-09-17T17:30:20.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000105807 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897079+0000", 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042597099999999998 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=32, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.702551662000001, 2024-09-17T17:30:20.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222760399, 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:20.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897503+0000", 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.68e-07 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897507+0000", 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7709999999999997e-06 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120082+0000", 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222574243 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120095+0000", 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.291e-05 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120199+0000", 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010402499999999999 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120262+0000", 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3781999999999993e-05 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 51/32 ECSubWrite(tid=61, reqid=osd.3.0:61, at_version=51'39, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.259932305, 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007939804, 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.632 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340122+0000", 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4200000000000005e-07 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340146+0000", 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3578e-05 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340199+0000", 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3127999999999997e-05 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340208+0000", 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3589999999999997e-06 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340279+0000", 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0772000000000006e-05 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340289+0000", 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9860000000000006e-06 2024-09-17T17:30:20.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347977+0000", 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076882779999999998 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348061+0000", 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3861000000000002e-05 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s1 51/35 ECSubWrite(tid=41, reqid=osd.10.0:41, at_version=51'30, trim_to=0'0, roll_forward_to=49'28))", 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.259729964999998, 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0077529670000000004, 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340350+0000", 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5931e-05 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340417+0000", 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6718999999999996e-05 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340426+0000", 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0489999999999993e-06 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340489+0000", 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3219000000000006e-05 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340498+0000", 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6740000000000003e-06 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348033+0000", 2024-09-17T17:30:20.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075347440000000003 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348077+0000", 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4076000000000003e-05 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 51/32 ECSubWrite(tid=62, reqid=osd.3.0:62, at_version=51'41, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.342655+0000", 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.257398918, 2024-09-17T17:30:20.641 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068192089999999997, 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342655+0000", 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342655+0000", 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342655+0000", 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2499999999999999e-07 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342709+0000", 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4194000000000003e-05 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342803+0000", 2024-09-17T17:30:20.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4226000000000002e-05 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342810+0000", 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8079999999999999e-06 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342954+0000", 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014373000000000001 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342964+0000", 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0348e-05 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349442+0000", 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0064773349999999999 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349474+0000", 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2342999999999998e-05 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.645 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s5 51/35 ECSubWrite(tid=121, reqid=osd.13.0:121, at_version=51'51, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.342707+0000", 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.257346359, 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0067213680000000001, 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342707+0000", 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342707+0000", 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342708+0000", 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9500000000000001e-07 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342736+0000", 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8161999999999999e-05 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342776+0000", 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9925999999999997e-05 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342789+0000", 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2873e-05 2024-09-17T17:30:20.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342938+0000", 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014885500000000001 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342948+0000", 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0567999999999999e-05 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349385+0000", 2024-09-17T17:30:20.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0064369199999999996 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349429+0000", 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3269000000000002e-05 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s5 51/35 ECSubWrite(tid=122, reqid=osd.13.0:122, at_version=51'53, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.343918+0000", 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.256135894, 2024-09-17T17:30:20.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079303870000000005, 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343918+0000", 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343918+0000", 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343919+0000", 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2079999999999999e-06 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343944+0000", 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4630999999999998e-05 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343977+0000", 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3506000000000001e-05 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343985+0000", 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2309999999999996e-06 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.346574+0000", 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0025890850000000001 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.346583+0000", 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7260000000000008e-06 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351805+0000", 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0052215819999999998 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351848+0000", 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3417999999999997e-05 2024-09-17T17:30:20.654 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:412 3.ds0 3:badfad2a:::smithi10238019-37:head [append 0~655360] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.243336851999999, 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097729180000000002, 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.655 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 412 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356719+0000", 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.255e-06 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357255+0000", 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00053548099999999998 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357270+0000", 2024-09-17T17:30:20.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5045000000000001e-05 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357278+0000", 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8429999999999999e-06 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357346+0000", 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7186000000000004e-05 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357460+0000", 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114388 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.358484+0000", 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023641 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366119+0000", 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076354270000000002 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366416+0000", 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00029701100000000002 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366490+0000", 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3640999999999995e-05 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.660 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:414 3.ds0 3:badfad2a:::smithi10238019-37:head [append 0~131072] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361831+0000", 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.238223113, 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.006203966, 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 414 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361831+0000", 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361831+0000", 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361832+0000", 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.018e-06 2024-09-17T17:30:20.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361881+0000", 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9549999999999998e-05 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361893+0000", 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1751e-05 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361898+0000", 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0769999999999998e-06 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.365013+0000", 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0031147089999999998 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.365122+0000", 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000109597 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.365754+0000", 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00063170700000000002 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367928+0000", 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0021745139999999998 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367972+0000", 2024-09-17T17:30:20.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3285999999999999e-05 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.368035+0000", 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2756999999999996e-05 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:415 3.ds0 3:badfad2a:::smithi10238019-37:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:20.666 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.238143190999999, 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089535759999999995, 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 415 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:20.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361911+0000", 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7e-07 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361913+0000", 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2350000000000002e-06 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361919+0000", 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4750000000000001e-06 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361922+0000", 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3110000000000001e-06 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366710+0000", 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047879840000000003 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366770+0000", 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9886000000000002e-05 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367045+0000", 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027536599999999998 2024-09-17T17:30:20.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370540+0000", 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034951610000000001 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370583+0000", 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2682999999999998e-05 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370864+0000", 2024-09-17T17:30:20.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00028100500000000002 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:416 3.ds0 3:badfad2a:::smithi10238019-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.238126165000001, 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096567449999999996, 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:20.672 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 416 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:20.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361928+0000", 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5400000000000002e-07 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361929+0000", 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.471e-06 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361934+0000", 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1100000000000002e-06 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361937+0000", 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1499999999999999e-06 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367591+0000", 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005653534 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367664+0000", 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2779000000000002e-05 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370895+0000", 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0032314409999999998 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370943+0000", 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7886999999999997e-05 2024-09-17T17:30:20.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.371584+0000", 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000641119 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s5 56/35 ECSubWrite(tid=143, reqid=osd.13.0:143, at_version=56'75, trim_to=0'0, roll_forward_to=53'73))", 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.340653+0000", 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 13.259400964999999, 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0062128360000000002, 2024-09-17T17:30:20.677 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340653+0000", 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340653+0000", 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340653+0000", 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4499999999999997e-07 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340666+0000", 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3188e-05 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340687+0000", 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1097000000000001e-05 2024-09-17T17:30:20.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340701+0000", 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3148e-05 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340743+0000", 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2091000000000001e-05 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.340751+0000", 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3629999999999994e-06 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.346824+0000", 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0060730209999999996 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.346866+0000", 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1483000000000002e-05 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 56/32 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=56'57, trim_to=0'0, roll_forward_to=53'55))", 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 13.256433993, 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089122520000000007, 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.92e-07 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343638+0000", 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8074e-05 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343672+0000", 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343680+0000", 2024-09-17T17:30:20.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8979999999999996e-06 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343804+0000", 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012350499999999999 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343814+0000", 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0353000000000001e-05 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352491+0000", 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086771880000000006 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352532+0000", 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0837999999999998e-05 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.bs5 57/32 ECSubWrite(tid=158, reqid=client.4566.0:492, at_version=0'0, trim_to=0'0, roll_forward_to=57'10))", 2024-09-17T17:30:20.686 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:08.657667+0000", 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.942387004, 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0059952390000000003, 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657667+0000", 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657667+0000", 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657668+0000", 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5600000000000004e-07 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657678+0000", 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0767999999999999e-05 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657748+0000", 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.9802999999999996e-05 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657760+0000", 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.145e-05 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657824+0000", 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4777000000000007e-05 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.657834+0000", 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.961e-06 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.663620+0000", 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057858400000000004 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.663662+0000", 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1783999999999998e-05 2024-09-17T17:30:20.690 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.16s5 68/67 ECSubWrite(tid=198, reqid=osd.13.0:198, at_version=68'119, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.463558+0000", 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.136495432, 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0077676009999999998, 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463558+0000", 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463558+0000", 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463559+0000", 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1900000000000001e-07 2024-09-17T17:30:20.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463583+0000", 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3561e-05 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463618+0000", 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5172000000000001e-05 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463625+0000", 2024-09-17T17:30:20.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1149999999999996e-06 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463687+0000", 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2220999999999999e-05 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463708+0000", 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1178999999999998e-05 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:20.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.471286+0000", 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075776359999999996 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.471326+0000", 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9898000000000003e-05 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:20.695 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:20.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:20.700 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:20.695+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:20.798 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:20.796+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:20.840 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:20.840 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:20.840 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:20.840 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:20.848 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:20.899 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:20.897+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:20.950 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:20.950 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:20.951 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:20.951 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:20.951 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:20.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:21.002 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:21.001+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:21.063 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:21.063 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=24, last_complete=36'10, committed=1, applied=1))", 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.606128402000003, 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53983539199999997, 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:21.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454674+0000", 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0790000000000002e-06 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992637+0000", 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53796236600000003 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992644+0000", 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5820000000000004e-06 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994469+0000", 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018243350000000001 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.067 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994505+0000", 2024-09-17T17:30:21.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6300999999999997e-05 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:173 3.cs0 3:30756a83:::smithi10238019-35:2 [read 297244~276196,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.070705768000003, 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011614354, 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.069 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 173 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990093+0000", 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990099+0000", 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2090000000000002e-06 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990111+0000", 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1372000000000001e-05 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990118+0000", 2024-09-17T17:30:21.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.126e-06 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990197+0000", 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8665999999999995e-05 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990299+0000", 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000102397 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001707+0000", 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011407690999999999 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:174 3.cs0 3:30756a83:::smithi10238019-35:2 [read 501714~71726,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.070660308999997, 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011875784, 2024-09-17T17:30:21.074 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 174 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9400000000000005e-07 2024-09-17T17:30:21.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990142+0000", 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.557e-06 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990195+0000", 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3012e-05 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990206+0000", 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0876000000000001e-05 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990783+0000", 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00057770200000000003 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990850+0000", 2024-09-17T17:30:21.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6168999999999996e-05 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.002014+0000", 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011163874000000001 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:175 3.cs0 3:30756a83:::smithi10238019-35:2 [read 194830~378610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:21.079 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.070573750999998, 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.30022850299999998, 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 175 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990225+0000", 2024-09-17T17:30:21.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8199999999999998e-07 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990230+0000", 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7809999999999996e-06 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990371+0000", 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014071299999999999 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990381+0000", 2024-09-17T17:30:21.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0061000000000001e-05 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994702+0000", 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043214899999999999 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994769+0000", 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6600999999999995e-05 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290453+0000", 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.29568397499999999 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=28, attrs_read=0))", 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.059600734999997, 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.289267987, 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.084 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:21.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001198+0000", 2024-09-17T17:30:21.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6000000000000004e-07 2024-09-17T17:30:21.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001279+0000", 2024-09-17T17:30:21.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1168000000000006e-05 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289843+0000", 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.28856357599999999 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289854+0000", 2024-09-17T17:30:21.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1749999999999999e-05 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289975+0000", 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012060100000000001 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290465+0000", 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049033200000000001 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.089 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:207 3.cs0 3:30d62bba:::smithi10238019-34:head [copy-from ver 2] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 30.354561669999999, 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011310328, 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.090 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 207 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706238+0000", 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9230000000000001e-06 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706245+0000", 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5089999999999999e-06 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706261+0000", 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6487999999999999e-05 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706268+0000", 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2859999999999996e-06 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706339+0000", 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1600999999999994e-05 2024-09-17T17:30:21.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706451+0000", 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000112071 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.714618+0000", 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0081665550000000007 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.715772+0000", 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011545959999999999 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.717464+0000", 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001691641 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.717547+0000", 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2657999999999994e-05 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.1s1 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.362909418000001, 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011087913, 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.096 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.55e-07 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697911+0000", 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1744999999999999e-05 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697975+0000", 2024-09-17T17:30:21.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4414999999999999e-05 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698108+0000", 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013266499999999999 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698174+0000", 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6011000000000002e-05 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698183+0000", 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2539999999999998e-06 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.708938+0000", 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010755602 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.708977+0000", 2024-09-17T17:30:21.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8065999999999998e-05 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:316 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (56) in=7b] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.07133997, 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041035350999999998, 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.101 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 316 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:21.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989459+0000", 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2200000000000004e-07 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989462+0000", 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.287e-06 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989469+0000", 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7229999999999998e-06 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989472+0000", 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.428e-06 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990430+0000", 2024-09-17T17:30:21.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00095784099999999997 2024-09-17T17:30:21.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990525+0000", 2024-09-17T17:30:21.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5129000000000003e-05 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990951+0000", 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042552399999999999 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991309+0000", 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000358632 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030441+0000", 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039131753999999998 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030493+0000", 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2510999999999999e-05 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:317 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:21.110 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.071306633999999, 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041694146000000001, 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 317 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989492+0000", 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3099999999999998e-07 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989494+0000", 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3329999999999999e-06 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989501+0000", 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.516e-06 2024-09-17T17:30:21.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989505+0000", 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.912e-06 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991167+0000", 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001662188 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:21.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991214+0000", 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6944000000000003e-05 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030733+0000", 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039519406999999999 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030782+0000", 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9338000000000001e-05 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.031186+0000", 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000403077 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:429 3.cs0 3:30d62bba:::smithi10238019-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.667895103999999, 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014137011, 2024-09-17T17:30:21.116 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 429 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:21.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3299999999999998e-07 2024-09-17T17:30:21.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392905+0000", 2024-09-17T17:30:21.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6869999999999999e-06 2024-09-17T17:30:21.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392912+0000", 2024-09-17T17:30:21.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6250000000000001e-06 2024-09-17T17:30:21.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392916+0000", 2024-09-17T17:30:21.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7330000000000003e-06 2024-09-17T17:30:21.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397110+0000", 2024-09-17T17:30:21.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041939450000000001 2024-09-17T17:30:21.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.123 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.123 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:21.123 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397209+0000", 2024-09-17T17:30:21.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8961000000000003e-05 2024-09-17T17:30:21.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402765+0000", 2024-09-17T17:30:21.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055556420000000004 2024-09-17T17:30:21.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402840+0000", 2024-09-17T17:30:21.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4943000000000004e-05 2024-09-17T17:30:21.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.407040+0000", 2024-09-17T17:30:21.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042001419999999996 2024-09-17T17:30:21.126 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.126 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.126 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.127 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.127 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:436 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 12] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:21.127 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:21.128 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.667798206000001, 2024-09-17T17:30:21.128 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012854290000000001, 2024-09-17T17:30:21.128 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.128 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.128 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.128 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.129 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.129 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.129 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 436 2024-09-17T17:30:21.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.129 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:21.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:21.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.132 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:21.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3700000000000001e-07 2024-09-17T17:30:21.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.132 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.132 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.132 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393005+0000", 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4190000000000004e-06 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393008+0000", 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7009999999999998e-06 2024-09-17T17:30:21.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.134 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.134 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393023+0000", 2024-09-17T17:30:21.134 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4545999999999999e-05 2024-09-17T17:30:21.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.134 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397903+0000", 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048805719999999997 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397964+0000", 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0520000000000003e-05 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.135 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.401036+0000", 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030720470000000001 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402722+0000", 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001685513 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405761+0000", 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030398790000000001 2024-09-17T17:30:21.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405854+0000", 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2756000000000004e-05 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:486 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 13.681791712999999, 2024-09-17T17:30:21.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75510421100000003, 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 486 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379007+0000", 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379011+0000", 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0720000000000001e-06 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379019+0000", 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9580000000000007e-06 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379025+0000", 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6799999999999998e-06 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379992+0000", 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096721299999999997 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380044+0000", 2024-09-17T17:30:21.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2091e-05 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380358+0000", 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000313543 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380731+0000", 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00037292 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134034+0000", 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75330350700000004 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134111+0000", 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6497999999999998e-05 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:487 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 13.681762572, 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.756014517, 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 487 2024-09-17T17:30:21.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.145 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379036+0000", 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5499999999999998e-07 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379038+0000", 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379045+0000", 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8770000000000004e-06 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379050+0000", 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2299999999999999e-06 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380522+0000", 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001471909 2024-09-17T17:30:21.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380575+0000", 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2821999999999997e-05 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134417+0000", 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75384171300000002 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134482+0000", 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5789000000000003e-05 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.135050+0000", 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00056751999999999996 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:501 3.cs0 3:3116c96c:::smithi10238019-11:head [stat,delete] snapc e=[e,d,9] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-17T17:30:21.149 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:10.332296+0000", 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.728502039, 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010718172, 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 501 2024-09-17T17:30:21.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332296+0000", 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332296+0000", 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332298+0000", 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9e-06 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332305+0000", 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.131e-06 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332322+0000", 2024-09-17T17:30:21.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6545000000000002e-05 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332330+0000", 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7860000000000002e-06 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332402+0000", 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1296000000000003e-05 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.332408+0000", 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9109999999999998e-06 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.338064+0000", 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0056561409999999999 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.338137+0000", 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3553000000000005e-05 2024-09-17T17:30:21.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:10.343014+0000", 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048769089999999996 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:626 3.cs0 3:30756a83:::smithi10238019-35:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.562500735, 2024-09-17T17:30:21.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021079339999999998, 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 626 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:21.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498298+0000", 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:21.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498301+0000", 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7530000000000002e-06 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498307+0000", 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9869999999999996e-06 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498310+0000", 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1259999999999998e-06 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499848+0000", 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001537912 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499903+0000", 2024-09-17T17:30:21.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5763000000000001e-05 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500222+0000", 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00031825499999999998 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.160 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500848+0000", 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062612400000000004 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519301+0000", 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018453431999999999 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.161 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519377+0000", 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5405000000000001e-05 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:627 3.cs0 3:30756a83:::smithi10238019-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.562471575, 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021540693, 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.162 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 627 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.163 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498327+0000", 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.51e-07 2024-09-17T17:30:21.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498329+0000", 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3989999999999998e-06 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498337+0000", 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5379999999999997e-06 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.165 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498341+0000", 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4349999999999999e-06 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500569+0000", 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002227934 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:21.166 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500629+0000", 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9721e-05 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519409+0000", 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018780192000000001 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.167 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519465+0000", 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6062e-05 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519867+0000", 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000401961 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.168 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s6 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.5601852699999998, 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26792300099999999, 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.169 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500614+0000", 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6700000000000002e-07 2024-09-17T17:30:21.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500747+0000", 2024-09-17T17:30:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013382499999999999 2024-09-17T17:30:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767249+0000", 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26650138800000001 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767261+0000", 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1785e-05 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767372+0000", 2024-09-17T17:30:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111743 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767387+0000", 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4548e-05 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768494+0000", 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011074990000000001 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.173 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768536+0000", 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1346e-05 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:639 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.537003264, 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67002614400000005, 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.174 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 639 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.175 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0500000000000003e-07 2024-09-17T17:30:21.176 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1729999999999999e-06 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523810+0000", 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1627000000000001e-05 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.177 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523815+0000", 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0420000000000002e-06 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525161+0000", 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0013460169999999999 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525253+0000", 2024-09-17T17:30:21.178 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1793999999999996e-05 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526004+0000", 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00075112400000000004 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527034+0000", 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001029713 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.179 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193749+0000", 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.66671533999999999 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193821+0000", 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1709000000000001e-05 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.180 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:640 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.536971648, 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67078859899999999, 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.181 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 640 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523827+0000", 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.03e-07 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523829+0000", 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1739999999999999e-06 2024-09-17T17:30:21.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523836+0000", 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3869999999999997e-06 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523841+0000", 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4089999999999996e-06 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.184 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527242+0000", 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003401383 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527298+0000", 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5705000000000003e-05 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.185 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194159+0000", 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.666861484 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194218+0000", 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9039000000000002e-05 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.186 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194615+0000", 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039661500000000001 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 68/32 ECSubWrite(tid=118, reqid=client.4566.0:641, at_version=68'91, trim_to=0'0, roll_forward_to=66'89))", 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.536130532, 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.164368816, 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.187 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524694+0000", 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6256000000000001e-05 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.189 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687716+0000", 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.163021575 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687725+0000", 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9390000000000003e-06 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687798+0000", 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900999999999999e-05 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.190 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687809+0000", 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.153e-05 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689007+0000", 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001197844 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689036+0000", 2024-09-17T17:30:21.191 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9342000000000001e-05 2024-09-17T17:30:21.192 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.192 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.192 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.192 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.192 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.192 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:21.194 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:21.194 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:21.192+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:21.294 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:21.293+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:21.337 INFO:tasks.rados.rados.0.smithi102.stdout:256: done (1 left) 2024-09-17T17:30:21.337 INFO:tasks.rados.rados.0.smithi102.stdout:257: snap_remove snap 15 2024-09-17T17:30:21.337 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:21.338 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:21.338 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:21.338 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:21.350 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:30:21.350 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:21.350 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-17T17:30:21.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T17:30:21.352 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:21.436 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:21.434+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:21.494 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:21.494 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:21.494 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:21.494 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:21.494 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:21.496 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:21.495+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:21.503 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:21.597 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:21.595+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:21.648 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:21.648 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:21.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:21.648 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:21.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.648 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 37/35 ECSubWriteReply(tid=17, last_complete=37'13, committed=1, applied=1))", 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.644502594999999, 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.068701408000000005, 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000937+0000", 2024-09-17T17:30:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6449999999999998e-06 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068896+0000", 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.067959031000000003 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068909+0000", 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3416e-05 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069008+0000", 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8690999999999994e-05 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069634+0000", 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062589099999999995 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 30.961571336999999, 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079875469999999994, 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:21.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2e-07 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684336+0000", 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047134800000000001 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684439+0000", 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010291800000000001 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684449+0000", 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.027e-05 2024-09-17T17:30:21.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684596+0000", 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000146749 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684607+0000", 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1421e-05 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691787+0000", 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071804069999999998 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691851+0000", 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3913999999999998e-05 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.657 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685294+0000", 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 30.960140865, 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0066107179999999998, 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685294+0000", 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685294+0000", 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685295+0000", 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.065e-06 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685313+0000", 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8029999999999998e-05 2024-09-17T17:30:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686226+0000", 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000912775 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686236+0000", 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4369999999999996e-06 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686307+0000", 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1019999999999994e-05 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.686316+0000", 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1719999999999996e-06 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691864+0000", 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055479930000000002 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691905+0000", 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1226000000000001e-05 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 30.960080931, 2024-09-17T17:30:21.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089994210000000005, 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4100000000000001e-07 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685373+0000", 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9063999999999999e-05 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687383+0000", 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002009355 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687391+0000", 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9699999999999999e-06 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687457+0000", 2024-09-17T17:30:21.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5834999999999996e-05 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9990000000000003e-06 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694321+0000", 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068546010000000001 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694354+0000", 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2456000000000002e-05 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:191 3.5s0 3:a4d61510:::smithi10238019-32:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 30.957998068999999, 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.008961732, 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.667 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 191 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687439+0000", 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1799999999999999e-06 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687504+0000", 2024-09-17T17:30:21.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4770000000000005e-05 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687516+0000", 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2007e-05 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687531+0000", 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4803e-05 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687595+0000", 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3697999999999999e-05 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687756+0000", 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000161846 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.690070+0000", 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0023137639999999998 2024-09-17T17:30:21.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695782+0000", 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057116959999999996 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695908+0000", 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000126339 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.696399+0000", 2024-09-17T17:30:21.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049062900000000002 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:407 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.307357920000001, 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011701008000000001, 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.673 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 407 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:21.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338080+0000", 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2819999999999999e-06 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6289999999999999e-06 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338108+0000", 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9514e-05 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338117+0000", 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.132e-06 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1446999999999995e-05 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338196+0000", 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6350000000000006e-06 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341655+0000", 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034593860000000001 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342199+0000", 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054402400000000005 2024-09-17T17:30:21.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349778+0000", 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075789589999999997 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:408 3.5s0 3:a96b502c:::smithi10238019-42:head [read 53220~651292,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.307195993000001, 2024-09-17T17:30:21.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012064669, 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 408 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338240+0000", 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338257+0000", 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6643e-05 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338270+0000", 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3081000000000001e-05 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338277+0000", 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3780000000000003e-06 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338331+0000", 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4286999999999997e-05 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338338+0000", 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3609999999999997e-06 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342681+0000", 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043432749999999997 2024-09-17T17:30:21.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342775+0000", 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4392e-05 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.350304+0000", 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007528435 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:409 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.307083238000001, 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012665852, 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 409 2024-09-17T17:30:21.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338353+0000", 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.09e-07 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338358+0000", 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7859999999999996e-06 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338366+0000", 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6049999999999998e-06 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338372+0000", 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1739999999999997e-06 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338439+0000", 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7039999999999995e-05 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338446+0000", 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1160000000000001e-06 2024-09-17T17:30:21.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343114+0000", 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0046672179999999999 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343207+0000", 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3170000000000004e-05 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351018+0000", 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078109340000000003 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 51/35 ECSubWrite(tid=122, reqid=osd.13.0:122, at_version=51'53, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.30139943, 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073809449999999999, 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.689 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1399999999999996e-07 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344057+0000", 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0273000000000001e-05 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344096+0000", 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9045000000000003e-05 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344104+0000", 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2050000000000002e-06 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344311+0000", 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00020691500000000001 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344321+0000", 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0307e-05 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351381+0000", 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0070600910000000001 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351417+0000", 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5395000000000002e-05 2024-09-17T17:30:21.693 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.292928858, 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.988933536, 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352507+0000", 2024-09-17T17:30:21.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1500000000000003e-07 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352532+0000", 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4485999999999999e-05 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340513+0000", 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.98798163100000003 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.696 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340522+0000", 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9379999999999998e-06 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340625+0000", 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010273 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340633+0000", 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341398+0000", 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00076522600000000004 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341440+0000", 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1548999999999998e-05 2024-09-17T17:30:21.698 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 51/35 ECSubWrite(tid=66, reqid=client.4566.0:410, at_version=0'0, trim_to=0'0, roll_forward_to=51'31))", 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356289+0000", 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.289146086999999, 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.006488148, 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.699 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356289+0000", 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356289+0000", 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356290+0000", 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2900000000000002e-07 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356299+0000", 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1919999999999994e-06 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356315+0000", 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5739999999999998e-05 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356321+0000", 2024-09-17T17:30:21.701 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1199999999999999e-06 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356371+0000", 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9697000000000002e-05 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356378+0000", 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0600000000000002e-06 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.702 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.362735+0000", 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063571859999999999 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.362777+0000", 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2324000000000003e-05 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/35 ECSubWrite(tid=50, reqid=client.4566.0:441, at_version=0'0, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:21.703 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.781871337999998, 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.121580543, 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.704 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863565+0000", 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6729999999999999e-06 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863588+0000", 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2756e-05 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984520+0000", 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.120931489 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984547+0000", 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6877999999999999e-05 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984631+0000", 2024-09-17T17:30:21.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4246000000000006e-05 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984638+0000", 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3509999999999996e-06 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985117+0000", 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047851700000000003 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985144+0000", 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7633000000000001e-05 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.0s2 53/35 ECSubWrite(tid=79, reqid=osd.12.0:79, at_version=53'28, trim_to=0'0, roll_forward_to=52'24))", 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.286973582000002, 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0064458709999999997, 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.708 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.709 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358462+0000", 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4199999999999996e-07 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358483+0000", 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0854000000000001e-05 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358510+0000", 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7033999999999999e-05 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358517+0000", 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2099999999999996e-06 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358568+0000", 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0621999999999997e-05 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.358576+0000", 2024-09-17T17:30:21.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0660000000000004e-06 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.364850+0000", 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0062745359999999998 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.364907+0000", 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7006999999999998e-05 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 53/32 ECSubWrite(tid=72, reqid=osd.3.0:72, at_version=53'53, trim_to=0'0, roll_forward_to=52'49))", 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.282887023000001, 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0070743660000000003, 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362549+0000", 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5400000000000001e-07 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362568+0000", 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9213000000000001e-05 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362606+0000", 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7647000000000001e-05 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362614+0000", 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2579999999999995e-06 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363252+0000", 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00063853300000000005 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363262+0000", 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9359999999999994e-06 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369572+0000", 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063095340000000003 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369622+0000", 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0590999999999997e-05 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 53/32 ECSubWrite(tid=34, reqid=osd.14.0:34, at_version=53'30, trim_to=0'0, roll_forward_to=52'28))", 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.282001271999999, 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0062252610000000002, 2024-09-17T17:30:21.717 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363434+0000", 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.68e-07 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363456+0000", 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2242000000000002e-05 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363643+0000", 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000186682 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363652+0000", 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5189999999999998e-06 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363740+0000", 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7960000000000005e-05 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363753+0000", 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2985e-05 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369627+0000", 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0058739049999999996 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369659+0000", 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.18e-05 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 2.1447808049999999, 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.090576745, 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.722 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500656+0000", 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0430000000000002e-06 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500772+0000", 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000115338 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590079+0000", 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089307485000000006 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590103+0000", 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4131e-05 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590159+0000", 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5417000000000002e-05 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590177+0000", 2024-09-17T17:30:21.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7819e-05 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591201+0000", 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023939 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591231+0000", 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0573e-05 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.726 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 70/32 ECSubWrite(tid=129, reqid=osd.3.0:129, at_version=70'99, trim_to=0'0, roll_forward_to=69'95))", 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:21.340542+0000", 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.30489359300000002, 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0060755740000000003, 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340542+0000", 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340542+0000", 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340542+0000", 2024-09-17T17:30:21.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6899999999999996e-07 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340572+0000", 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9309999999999999e-05 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340618+0000", 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5992000000000003e-05 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.729 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340626+0000", 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0669999999999992e-06 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340684+0000", 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8088999999999999e-05 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.340708+0000", 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3866000000000002e-05 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.346589+0000", 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0058810449999999997 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.346617+0000", 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8436e-05 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.731 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 70/67 ECSubWrite(tid=138, reqid=osd.12.0:138, at_version=70'53, trim_to=0'0, roll_forward_to=68'51))", 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:21.342419+0000", 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.30301657999999998, 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068936620000000001, 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342419+0000", 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342419+0000", 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342419+0000", 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7700000000000001e-07 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342471+0000", 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1746999999999998e-05 2024-09-17T17:30:21.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342504+0000", 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3081999999999999e-05 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342512+0000", 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.14e-06 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342547+0000", 2024-09-17T17:30:21.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5114000000000003e-05 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.342552+0000", 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2079999999999999e-06 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.349242+0000", 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0066900939999999997 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.349312+0000", 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.9999999999999994e-05 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.18s5 70/67 ECSubWrite(tid=154, reqid=osd.6.0:154, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:21.345187+0000", 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.300248442, 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0069789199999999996, 2024-09-17T17:30:21.736 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345187+0000", 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345187+0000", 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345187+0000", 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.34e-07 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345207+0000", 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9746999999999999e-05 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345324+0000", 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000117767 2024-09-17T17:30:21.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345333+0000", 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7940000000000008e-06 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345410+0000", 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6308000000000002e-05 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.739 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345423+0000", 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3937000000000001e-05 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.352122+0000", 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0066986440000000001 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.352166+0000", 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3489000000000003e-05 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.8s5 70/67 ECSubWrite(tid=155, reqid=osd.6.0:155, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:21.345209+0000", 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.30022639699999998, 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0069099879999999997, 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.741 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345209+0000", 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345209+0000", 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345209+0000", 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:30:21.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345230+0000", 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0514e-05 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345483+0000", 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00025305899999999998 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345492+0000", 2024-09-17T17:30:21.743 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4879999999999996e-06 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345545+0000", 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3162999999999999e-05 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.345551+0000", 2024-09-17T17:30:21.744 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0249999999999999e-06 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.352076+0000", 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0065251670000000001 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:21.352119+0000", 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2271999999999998e-05 2024-09-17T17:30:21.745 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.746 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.746 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.746 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:21.746 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:21.746 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:21.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:21.747 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T17:30:21.673+0000 7f873ad1c640 -1 mon.a@0(leader).osd e70 definitely_dead 0 2024-09-17T17:30:21.748 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:21.746+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:21.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:21.848 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:21.847+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:21.857 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:21.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:21.884 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:21.907 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:21.929 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:21.949 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:21.947+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:21.951 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:21.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:21.971 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:21.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:21.977 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:21.977 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:21.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:21.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:22.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:22.038 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:22.049 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:22.048+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:22.058 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:22.080 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:22.080 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:22.102 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:22.102 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:22.102 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.103 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:22.123 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:22.145 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:22.146 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:22.150 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:22.149+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:22.231 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.231 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:22.251 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:22.249+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:22.336 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.2. 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 6 to smithi10238019-36 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: finishing write tid 7 to smithi10238019-36 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: oid 36 updating version 0 to 56 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: oid 36 updating version 56 to 58 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: oid 36 version 58 is already newer than 54 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: oid 36 version 58 is already newer than 55 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: oid 36 version 58 is already newer than 57 2024-09-17T17:30:22.339 INFO:tasks.rados.rados.0.smithi102.stdout:249: oid 36 version 58 is already newer than 53 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 36 v 58 (ObjNum 119 snap 19 seq_num 119) dirty exists 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:249: left oid 36 (ObjNum 119 snap 19 seq_num 119) 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:249: done (1 left) 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:257: done (0 left) 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:258: rollback oid 29 current snap is 19 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 29 to 17 2024-09-17T17:30:22.340 INFO:tasks.rados.rados.0.smithi102.stdout:259: delete oid 32 current snap is 19 2024-09-17T17:30:22.351 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:22.349+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:22.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:30:22.356 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-17T17:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:22.370+0000 7f9e5affd640 0 -- 172.21.15.7:0/2087920338 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f9e5c21f2d0 msgr2=0x7f9e5c2216c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:22.453 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:22.451+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:22.554 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:22.552+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:22.556 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.556 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:22.595 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:22.616 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:22.669 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:22.667+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:22.674 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.674 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:22.755 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:22.754+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:22.830 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.830 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:22.857 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:22.855+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:22.946 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:22.946 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:22.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:22.958 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:22.956+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:23.058 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:23.056+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:23.062 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.062 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:23.159 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:23.157+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:23.218 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.218 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:23.260 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:23.259+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:23.337 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:30:23.361 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:23.360+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:23.381 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.381 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:23.462 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:23.460+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:23.512 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.512 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:23.565 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:23.564+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:23.629 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.629 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:23.667 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:23.665+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:23.744 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.745 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:23.768 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:23.767+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:23.861 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:23.861 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:23.868 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:23.867+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:23.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:23.970 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:23.968+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:24.017 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.018 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:24.110 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:24.108+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:24.171 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:24.170+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:24.173 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.173 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:24.272 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:24.270+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:24.330 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.330 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:24.368 INFO:tasks.rados.rados.0.smithi102.stdout:259: done (1 left) 2024-09-17T17:30:24.368 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 15 current snap is 19 2024-09-17T17:30:24.368 INFO:tasks.rados.rados.0.smithi102.stdout:260: seq_num 122 ranges {1622016=507904,2129920=425984,2555904=442368,2998272=540672,3538944=147456} 2024-09-17T17:30:24.372 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:24.371+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:24.380 INFO:tasks.rados.rados.0.smithi102.stdout:260: writing smithi10238019-15 from 1622016 to 2129920 tid 1 2024-09-17T17:30:24.382 INFO:tasks.rados.rados.0.smithi102.stdout:260: writing smithi10238019-15 from 2129920 to 2555904 tid 2 2024-09-17T17:30:24.388 INFO:tasks.rados.rados.0.smithi102.stdout:260: writing smithi10238019-15 from 2555904 to 2998272 tid 3 2024-09-17T17:30:24.389 INFO:tasks.rados.rados.0.smithi102.stdout:260: writing smithi10238019-15 from 2998272 to 3538944 tid 4 2024-09-17T17:30:24.390 INFO:tasks.rados.rados.0.smithi102.stdout:260: writing smithi10238019-15 from 3538944 to 3686400 tid 5 2024-09-17T17:30:24.390 INFO:tasks.rados.rados.0.smithi102.stdout:261: read oid 31 snap -1 2024-09-17T17:30:24.390 INFO:tasks.rados.rados.0.smithi102.stdout:261: expect deleted 2024-09-17T17:30:24.390 INFO:tasks.rados.rados.0.smithi102.stdout:262: delete oid 45 current snap is 19 2024-09-17T17:30:24.390 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 1 to smithi10238019-15 2024-09-17T17:30:24.390 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 2 to smithi10238019-15 2024-09-17T17:30:24.396 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 3 to smithi10238019-15 2024-09-17T17:30:24.396 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 4 to smithi10238019-15 2024-09-17T17:30:24.400 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 5 to smithi10238019-15 2024-09-17T17:30:24.400 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 6 to smithi10238019-15 2024-09-17T17:30:24.402 INFO:tasks.rados.rados.0.smithi102.stdout:260: finishing write tid 7 to smithi10238019-15 2024-09-17T17:30:24.402 INFO:tasks.rados.rados.0.smithi102.stdout:260: oid 15 updating version 0 to 55 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:260: oid 15 version 55 is already newer than 52 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:260: oid 15 version 55 is already newer than 51 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:260: oid 15 version 55 is already newer than 50 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:260: oid 15 version 55 is already newer than 53 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:260: oid 15 version 55 is already newer than 54 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 15 v 55 (ObjNum 122 snap 19 seq_num 122) dirty exists 2024-09-17T17:30:24.403 INFO:tasks.rados.rados.0.smithi102.stdout:260: left oid 15 (ObjNum 122 snap 19 seq_num 122) 2024-09-17T17:30:24.473 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:24.472+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:24.489 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.490 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.498 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:24.574 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:24.572+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:24.607 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.607 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:24.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:24.649 INFO:tasks.rados.rados.0.smithi102.stdout:260: done (3 left) 2024-09-17T17:30:24.650 INFO:tasks.rados.rados.0.smithi102.stdout:261: done (2 left) 2024-09-17T17:30:24.650 INFO:tasks.rados.rados.0.smithi102.stdout:262: done (1 left) 2024-09-17T17:30:24.650 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 7 current snap is 19 2024-09-17T17:30:24.650 INFO:tasks.rados.rados.0.smithi102.stdout:263: seq_num 124 ranges {2686976=540672,3227648=720896,3948544=786432,4734976=49152} 2024-09-17T17:30:24.667 INFO:tasks.rados.rados.0.smithi102.stdout:263: writing smithi10238019-7 from 2686976 to 3227648 tid 1 2024-09-17T17:30:24.671 INFO:tasks.rados.rados.0.smithi102.stdout:263: writing smithi10238019-7 from 3227648 to 3948544 tid 2 2024-09-17T17:30:24.675 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:24.674+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:24.676 INFO:tasks.rados.rados.0.smithi102.stdout:263: writing smithi10238019-7 from 3948544 to 4734976 tid 3 2024-09-17T17:30:24.676 INFO:tasks.rados.rados.0.smithi102.stdout:263: writing smithi10238019-7 from 4734976 to 4784128 tid 4 2024-09-17T17:30:24.677 INFO:tasks.rados.rados.0.smithi102.stdout:264: setattr oid 11 current snap is 19 2024-09-17T17:30:24.677 INFO:tasks.rados.rados.0.smithi102.stdout:263: finishing write tid 1 to smithi10238019-7 2024-09-17T17:30:24.678 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 32 current snap is 19 2024-09-17T17:30:24.678 INFO:tasks.rados.rados.0.smithi102.stdout:265: seq_num 125 ranges {0=557056,557056=475136,1032192=507904,1540096=475136,2015232=98304} 2024-09-17T17:30:24.682 INFO:tasks.rados.rados.0.smithi102.stdout:265: writing smithi10238019-32 from 0 to 557056 tid 1 2024-09-17T17:30:24.685 INFO:tasks.rados.rados.0.smithi102.stdout:265: writing smithi10238019-32 from 557056 to 1032192 tid 2 2024-09-17T17:30:24.688 INFO:tasks.rados.rados.0.smithi102.stdout:265: writing smithi10238019-32 from 1032192 to 1540096 tid 3 2024-09-17T17:30:24.691 INFO:tasks.rados.rados.0.smithi102.stdout:265: writing smithi10238019-32 from 1540096 to 2015232 tid 4 2024-09-17T17:30:24.692 INFO:tasks.rados.rados.0.smithi102.stdout:265: writing smithi10238019-32 from 2015232 to 2113536 tid 5 2024-09-17T17:30:24.692 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 21 current snap is 19 2024-09-17T17:30:24.692 INFO:tasks.rados.rados.0.smithi102.stdout:266: seq_num 126 ranges {2097152=573440,2670592=622592,3293184=622592,3915776=311296} 2024-09-17T17:30:24.707 INFO:tasks.rados.rados.0.smithi102.stdout:266: writing smithi10238019-21 from 2097152 to 2670592 tid 1 2024-09-17T17:30:24.710 INFO:tasks.rados.rados.0.smithi102.stdout:266: writing smithi10238019-21 from 2670592 to 3293184 tid 2 2024-09-17T17:30:24.713 INFO:tasks.rados.rados.0.smithi102.stdout:266: writing smithi10238019-21 from 3293184 to 3915776 tid 3 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:266: writing smithi10238019-21 from 3915776 to 4227072 tid 4 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:267: read oid 22 snap 17 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:267: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:268: copy_from oid 9 from oid 49 current snap is 19 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:263: finishing write tid 2 to smithi10238019-7 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:263: finishing write tid 3 to smithi10238019-7 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:269: read oid 10 snap 17 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:269: expect (ObjNum 102 snap 13 seq_num 102) 2024-09-17T17:30:24.716 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 47 current snap is 19 2024-09-17T17:30:24.717 INFO:tasks.rados.rados.0.smithi102.stdout:270: seq_num 127 ranges {1851392=589824,2441216=753664,3194880=475136,3670016=327680} 2024-09-17T17:30:24.729 INFO:tasks.rados.rados.0.smithi102.stdout:270: writing smithi10238019-47 from 1851392 to 2441216 tid 1 2024-09-17T17:30:24.734 INFO:tasks.rados.rados.0.smithi102.stdout:270: writing smithi10238019-47 from 2441216 to 3194880 tid 2 2024-09-17T17:30:24.737 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.737 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.737 INFO:tasks.rados.rados.0.smithi102.stdout:270: writing smithi10238019-47 from 3194880 to 3670016 tid 3 2024-09-17T17:30:24.740 INFO:tasks.rados.rados.0.smithi102.stdout:270: writing smithi10238019-47 from 3670016 to 3997696 tid 4 2024-09-17T17:30:24.740 INFO:tasks.rados.rados.0.smithi102.stdout:271: setattr oid 27 current snap is 19 2024-09-17T17:30:24.740 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 11 v 120 (ObjNum 111 snap 17 seq_num 111) dirty exists 2024-09-17T17:30:24.740 INFO:tasks.rados.rados.0.smithi102.stdout:263: finishing write tid 4 to smithi10238019-7 2024-09-17T17:30:24.740 INFO:tasks.rados.rados.0.smithi102.stdout:263: finishing write tid 5 to smithi10238019-7 2024-09-17T17:30:24.740 INFO:tasks.rados.rados.0.smithi102.stdout:263: finishing write tid 6 to smithi10238019-7 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:263: oid 7 updating version 0 to 59 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:263: oid 7 updating version 59 to 62 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:263: oid 7 version 62 is already newer than 61 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:263: oid 7 version 62 is already newer than 60 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:263: oid 7 version 62 is already newer than 58 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 62 (ObjNum 124 snap 19 seq_num 124) dirty exists 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:263: left oid 7 (ObjNum 124 snap 19 seq_num 124) 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 1 to smithi10238019-32 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 2 to smithi10238019-32 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 3 to smithi10238019-32 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 4 to smithi10238019-32 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 5 to smithi10238019-32 2024-09-17T17:30:24.741 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 6 to smithi10238019-32 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: finishing write tid 7 to smithi10238019-32 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: oid 32 updating version 0 to 25 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: oid 32 updating version 25 to 26 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: oid 32 updating version 26 to 27 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: oid 32 version 27 is already newer than 24 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: oid 32 version 27 is already newer than 23 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: oid 32 version 27 is already newer than 22 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 32 v 27 (ObjNum 125 snap 19 seq_num 125) dirty exists 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:265: left oid 32 (ObjNum 125 snap 19 seq_num 125) 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:266: finishing write tid 1 to smithi10238019-21 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:266: finishing write tid 2 to smithi10238019-21 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:266: finishing write tid 3 to smithi10238019-21 2024-09-17T17:30:24.742 INFO:tasks.rados.rados.0.smithi102.stdout:266: finishing write tid 4 to smithi10238019-21 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: finishing write tid 5 to smithi10238019-21 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: finishing write tid 6 to smithi10238019-21 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: oid 21 updating version 0 to 46 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: oid 21 version 46 is already newer than 45 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: oid 21 updating version 46 to 48 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: oid 21 version 48 is already newer than 47 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: oid 21 version 48 is already newer than 44 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 21 v 48 (ObjNum 126 snap 19 seq_num 126) dirty exists 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:266: left oid 21 (ObjNum 126 snap 19 seq_num 126) 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:268: finishing copy_from racing read to smithi10238019-9 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:268: finishing copy_from to smithi10238019-9 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-17T17:30:24.743 INFO:tasks.rados.rados.0.smithi102.stdout:270: finishing write tid 1 to smithi10238019-47 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:263: done (9 left) 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:264: done (8 left) 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:265: done (7 left) 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:266: done (6 left) 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:268: done (5 left) 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:272: read oid 28 snap -1 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:272: expect deleted 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:273: read oid 5 snap -1 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:273: expect (ObjNum 107 snap 16 seq_num 107) 2024-09-17T17:30:24.745 INFO:tasks.rados.rados.0.smithi102.stdout:274: copy_from oid 6 from oid 31 current snap is 19 2024-09-17T17:30:24.746 INFO:tasks.rados.rados.0.smithi102.stdout:275: snap_create 2024-09-17T17:30:24.746 INFO:tasks.rados.rados.0.smithi102.stdout:270: finishing write tid 2 to smithi10238019-47 2024-09-17T17:30:24.746 INFO:tasks.rados.rados.0.smithi102.stdout:270: finishing write tid 3 to smithi10238019-47 2024-09-17T17:30:24.746 INFO:tasks.rados.rados.0.smithi102.stdout:270: finishing write tid 4 to smithi10238019-47 2024-09-17T17:30:24.746 INFO:tasks.rados.rados.0.smithi102.stdout:258: finishing rollback tid 0 to smithi10238019-29 2024-09-17T17:30:24.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:24.749 INFO:tasks.rados.rados.0.smithi102.stdout:274: finishing copy_from to smithi10238019-6 2024-09-17T17:30:24.750 INFO:tasks.rados.rados.0.smithi102.stdout:274: got expected ENOENT (src dne) 2024-09-17T17:30:24.750 INFO:tasks.rados.rados.0.smithi102.stdout:274: finishing copy_from racing read to smithi10238019-6 2024-09-17T17:30:24.764 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 27 v 52 (ObjNum 87 snap 10 seq_num 87) dirty exists 2024-09-17T17:30:24.770 INFO:tasks.rados.rados.0.smithi102.stdout:273: expect (ObjNum 107 snap 16 seq_num 107) 2024-09-17T17:30:24.776 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:24.774+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:24.779 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1680287 2024-09-17T17:30:24.782 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 406384 2024-09-17T17:30:24.783 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:24.804 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1723035 2024-09-17T17:30:24.805 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:24.814 INFO:tasks.rados.rados.0.smithi102.stdout:258: finishing rollback tid 1 to smithi10238019-29 2024-09-17T17:30:24.815 INFO:tasks.rados.rados.0.smithi102.stdout:258: finishing rollback tid 2 to smithi10238019-29 2024-09-17T17:30:24.815 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 29 v 62 (ObjNum 93 snap 11 seq_num 93) dirty exists 2024-09-17T17:30:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:24.829 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:24.845 INFO:tasks.rados.rados.0.smithi102.stdout:269: expect (ObjNum 102 snap 13 seq_num 102) 2024-09-17T17:30:24.845 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:24.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:24.855 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.856 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:24.867 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 799781 2024-09-17T17:30:24.870 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:24.871 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:24.876 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:24.877 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:24.875+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:24.892 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:24.892 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:24.898 INFO:tasks.rados.rados.0.smithi102.stdout:267: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-17T17:30:24.899 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 99855 2024-09-17T17:30:24.903 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:24.911 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 475458 2024-09-17T17:30:24.913 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:24.934 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:24.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:24.971 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:24.971 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:24.975 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:24.978 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:24.977+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:24.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:24.998 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:24.999 INFO:tasks.rados.rados.0.smithi102.stdout:270: finishing write tid 5 to smithi10238019-47 2024-09-17T17:30:25.001 INFO:tasks.rados.rados.0.smithi102.stdout:270: finishing write tid 6 to smithi10238019-47 2024-09-17T17:30:25.001 INFO:tasks.rados.rados.0.smithi102.stdout:270: oid 47 updating version 0 to 104 2024-09-17T17:30:25.001 INFO:tasks.rados.rados.0.smithi102.stdout:270: oid 47 updating version 104 to 106 2024-09-17T17:30:25.001 INFO:tasks.rados.rados.0.smithi102.stdout:270: oid 47 updating version 106 to 108 2024-09-17T17:30:25.004 INFO:tasks.rados.rados.0.smithi102.stdout:270: oid 47 version 108 is already newer than 107 2024-09-17T17:30:25.005 INFO:tasks.rados.rados.0.smithi102.stdout:270: oid 47 version 108 is already newer than 105 2024-09-17T17:30:25.005 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 47 v 108 (ObjNum 127 snap 19 seq_num 127) dirty exists 2024-09-17T17:30:25.005 INFO:tasks.rados.rados.0.smithi102.stdout:270: left oid 47 (ObjNum 127 snap 19 seq_num 127) 2024-09-17T17:30:25.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:25.021 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:25.041 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:25.041 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:25.062 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:25.066+0000 7f71b3fff640 0 -- 172.21.15.7:0/374273623 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f71b421f340 msgr2=0x7f71b4221730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:25.078 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:25.076+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:25.089 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:25.090 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:25.091 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:25.091 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:25.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:25.110 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:25.179 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:25.178+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:25.206 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:25.206 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:25.206 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:25.206 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:25.206 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:25.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:25.280 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:25.278+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:25.327 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.13s5 73/72 [PushOp(3:c80ea1ca:::smithi10238019-27:head, version: 73'52, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 24, recovery_info: ObjectRecoveryInfo(3:c80ea1ca:::smithi10238019-27:head@73'52, size: 1785856, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:24.834501+0000", 2024-09-17T17:30:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.48951168499999997, 2024-09-17T17:30:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066878200000000004, 2024-09-17T17:30:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834501+0000", 2024-09-17T17:30:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834501+0000", 2024-09-17T17:30:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834502+0000", 2024-09-17T17:30:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7599999999999997e-07 2024-09-17T17:30:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834669+0000", 2024-09-17T17:30:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016679999999999999 2024-09-17T17:30:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834701+0000", 2024-09-17T17:30:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2496999999999997e-05 2024-09-17T17:30:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834711+0000", 2024-09-17T17:30:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6260000000000007e-06 2024-09-17T17:30:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.834768+0000", 2024-09-17T17:30:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7362999999999999e-05 2024-09-17T17:30:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.835170+0000", 2024-09-17T17:30:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040192 2024-09-17T17:30:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:93d1baed:::smithi10238019-5:13, version: 68'45, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:93d1baed:::smithi10238019-5:13@68'45, size: 1802240, copy_subset: [], clone_subset: {}, snapset: 13={13: [13]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:24.984827+0000", 2024-09-17T17:30:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.33918613399999997, 2024-09-17T17:30:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00065723300000000001, 2024-09-17T17:30:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.984827+0000", 2024-09-17T17:30:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.984827+0000", 2024-09-17T17:30:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.984828+0000", 2024-09-17T17:30:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1349999999999999e-06 2024-09-17T17:30:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.985070+0000", 2024-09-17T17:30:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000242594 2024-09-17T17:30:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.985102+0000", 2024-09-17T17:30:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1714000000000002e-05 2024-09-17T17:30:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.985113+0000", 2024-09-17T17:30:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0721e-05 2024-09-17T17:30:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.985176+0000", 2024-09-17T17:30:25.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2661000000000002e-05 2024-09-17T17:30:25.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.985484+0000", 2024-09-17T17:30:25.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030840799999999998 2024-09-17T17:30:25.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.347 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.348 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:93d1baed:::smithi10238019-5:13, version: 68'45, data_included: [262144~188416], data_size: 188416, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:93d1baed:::smithi10238019-5:13@68'45, size: 1802240, copy_subset: [], clone_subset: {}, snapset: 13={13: [13]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1802240, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.348 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:24.989193+0000", 2024-09-17T17:30:25.349 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.33481950900000002, 2024-09-17T17:30:25.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069386600000000001, 2024-09-17T17:30:25.349 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.350 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989193+0000", 2024-09-17T17:30:25.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989193+0000", 2024-09-17T17:30:25.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989194+0000", 2024-09-17T17:30:25.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1999999999999999e-07 2024-09-17T17:30:25.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989373+0000", 2024-09-17T17:30:25.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017851499999999999 2024-09-17T17:30:25.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989402+0000", 2024-09-17T17:30:25.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9198999999999999e-05 2024-09-17T17:30:25.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989412+0000", 2024-09-17T17:30:25.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0642000000000001e-05 2024-09-17T17:30:25.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989496+0000", 2024-09-17T17:30:25.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3237000000000003e-05 2024-09-17T17:30:25.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.989887+0000", 2024-09-17T17:30:25.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000391553 2024-09-17T17:30:25.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.358 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:93d1baed:::smithi10238019-5:head, version: 68'46, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 24, recovery_info: ObjectRecoveryInfo(3:93d1baed:::smithi10238019-5:head@68'46, size: 1802240, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.358 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:24.992972+0000", 2024-09-17T17:30:25.358 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.331040628, 2024-09-17T17:30:25.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00074071, 2024-09-17T17:30:25.359 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.359 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.992972+0000", 2024-09-17T17:30:25.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.992972+0000", 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.992973+0000", 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.92e-07 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.993276+0000", 2024-09-17T17:30:25.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030347700000000002 2024-09-17T17:30:25.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.993305+0000", 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8784000000000001e-05 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.993316+0000", 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0777e-05 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.993382+0000", 2024-09-17T17:30:25.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5841999999999997e-05 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.993713+0000", 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033113800000000003 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:93d1baed:::smithi10238019-5:head, version: 68'46, data_included: [262144~188416], data_size: 188416, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:93d1baed:::smithi10238019-5:head@68'46, size: 1802240, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1802240, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:24.998208+0000", 2024-09-17T17:30:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.32580468699999998, 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00067558099999999997, 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998208+0000", 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998208+0000", 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998209+0000", 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0299999999999998e-07 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998391+0000", 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000182265 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998429+0000", 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8185000000000001e-05 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998446+0000", 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6892999999999999e-05 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998519+0000", 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2738999999999996e-05 2024-09-17T17:30:25.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:24.998884+0000", 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036479600000000002 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:96cbaefd:::smithi10238019-46:13, version: 68'47, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 23, recovery_info: ObjectRecoveryInfo(3:96cbaefd:::smithi10238019-46:13@68'47, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 13={10: [10], 13: [13, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.001697+0000", 2024-09-17T17:30:25.368 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.32231599100000002, 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00083606900000000005, 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.001697+0000", 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.001697+0000", 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.001702+0000", 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.412e-06 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.002024+0000", 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032171700000000001 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.002056+0000", 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1775e-05 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.002075+0000", 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8907999999999999e-05 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.002156+0000", 2024-09-17T17:30:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1130000000000004e-05 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.002533+0000", 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000377127 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:96cbaefd:::smithi10238019-46:13, version: 68'47, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:96cbaefd:::smithi10238019-46:13@68'47, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 13={10: [10], 13: [13, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.372 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.004765+0000", 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.319247485, 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064476500000000005, 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.004765+0000", 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.004765+0000", 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.004766+0000", 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1099999999999995e-07 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.005014+0000", 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024831199999999999 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.005043+0000", 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8662000000000001e-05 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.005053+0000", 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0284e-05 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.005120+0000", 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6587999999999993e-05 2024-09-17T17:30:25.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.005410+0000", 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029020800000000002 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:96cbaefd:::smithi10238019-46:13, version: 68'47, data_included: [524288~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:96cbaefd:::smithi10238019-46:13@68'47, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 13={10: [10], 13: [13, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.007568+0000", 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.316444371, 2024-09-17T17:30:25.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00065975300000000001, 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007568+0000", 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007568+0000", 2024-09-17T17:30:25.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007569+0000", 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6700000000000003e-07 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007818+0000", 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024848099999999999 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007846+0000", 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8251000000000001e-05 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007862+0000", 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6138999999999999e-05 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.007926+0000", 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3897000000000002e-05 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.008228+0000", 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030241799999999999 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:96cbaefd:::smithi10238019-46:10, version: 70'52, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:96cbaefd:::smithi10238019-46:10@70'52, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 13={10: [10], 13: [13, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.013403+0000", 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.310610096, 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064531300000000001, 2024-09-17T17:30:25.380 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013403+0000", 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013403+0000", 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013403+0000", 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7199999999999999e-07 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013656+0000", 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000252895 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013683+0000", 2024-09-17T17:30:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6605e-05 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013698+0000", 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4868000000000001e-05 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.013764+0000", 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6721e-05 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.014048+0000", 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028365200000000001 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:96cbaefd:::smithi10238019-46:10, version: 70'52, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:96cbaefd:::smithi10238019-46:10@70'52, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 13={10: [10], 13: [13, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.017573+0000", 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.30643964400000001, 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060107200000000002, 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017573+0000", 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017573+0000", 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017574+0000", 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3000000000000001e-07 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017841+0000", 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026755399999999998 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017867+0000", 2024-09-17T17:30:25.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5686000000000001e-05 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017882+0000", 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4902000000000001e-05 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.017947+0000", 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4979000000000002e-05 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.018174+0000", 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000227421 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s1 73/72 [PushOp(3:96cbaefd:::smithi10238019-46:10, version: 70'52, data_included: [524288~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:96cbaefd:::smithi10238019-46:10@70'52, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 13={10: [10], 13: [13, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.025228+0000", 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.29878439600000001, 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00058965099999999996, 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025228+0000", 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025228+0000", 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025229+0000", 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3099999999999997e-07 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025454+0000", 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000224503 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025483+0000", 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9781000000000002e-05 2024-09-17T17:30:25.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025501+0000", 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7931000000000002e-05 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025574+0000", 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2368999999999998e-05 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.025818+0000", 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000244336 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.0s3 73/72 [PushOp(3:00c2d3b1:::smithi10238019-29:head, version: 73'62, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:00c2d3b1:::smithi10238019-29:head@73'62, size: 1785856, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.122524+0000", 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.20148933399999999, 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00077858099999999998, 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122524+0000", 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122524+0000", 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122524+0000", 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122774+0000", 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024959299999999999 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122813+0000", 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8862000000000002e-05 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122827+0000", 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4068e-05 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.122930+0000", 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103429 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.123302+0000", 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037181200000000001 2024-09-17T17:30:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:682da707:::smithi10238019-8:head, version: 62'103, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:682da707:::smithi10238019-8:head@62'103, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.211441+0000", 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.11257170800000001, 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00096162999999999999, 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211441+0000", 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211441+0000", 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211442+0000", 2024-09-17T17:30:25.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000001e-07 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211765+0000", 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032307599999999999 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211804+0000", 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9221999999999998e-05 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211832+0000", 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7702e-05 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.211930+0000", 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8363999999999995e-05 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.212403+0000", 2024-09-17T17:30:25.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047240200000000002 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:682da707:::smithi10238019-8:head, version: 62'103, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:682da707:::smithi10238019-8:head@62'103, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.402 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.214639+0000", 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.10937411900000001, 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061013199999999999, 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214639+0000", 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214639+0000", 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214639+0000", 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.44e-07 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214882+0000", 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024274999999999999 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214911+0000", 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8714e-05 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214921+0000", 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8749999999999995e-06 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.214983+0000", 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2632000000000006e-05 2024-09-17T17:30:25.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.215249+0000", 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000265517 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:682da707:::smithi10238019-8:head, version: 62'103, data_included: [524288~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:682da707:::smithi10238019-8:head@62'103, size: 3309568, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.217262+0000", 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.106751279, 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00056190999999999999, 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217262+0000", 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217262+0000", 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217262+0000", 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-07 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217490+0000", 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022812700000000001 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217518+0000", 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.777e-05 2024-09-17T17:30:25.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217528+0000", 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6320000000000003e-06 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217583+0000", 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4923000000000003e-05 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.217824+0000", 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024086100000000001 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:6f83af53:::smithi10238019-50:13, version: 70'122, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:6f83af53:::smithi10238019-50:13@70'122, size: 819200, copy_subset: [], clone_subset: {}, snapset: 13={13: [13, 11, 10]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 819200, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.413 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.224542+0000", 2024-09-17T17:30:25.416 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.099470599000000007, 2024-09-17T17:30:25.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086491300000000003, 2024-09-17T17:30:25.416 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224542+0000", 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224542+0000", 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224543+0000", 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2600000000000002e-07 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224733+0000", 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019015199999999999 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224762+0000", 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8680999999999999e-05 2024-09-17T17:30:25.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224772+0000", 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0349999999999999e-05 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.224861+0000", 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8573999999999996e-05 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.225407+0000", 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054653000000000004 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:6f83af53:::smithi10238019-50:head, version: 70'123, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 24, recovery_info: ObjectRecoveryInfo(3:6f83af53:::smithi10238019-50:head@70'123, size: 819200, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 819200, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.228112+0000", 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.095900509999999994, 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00072902999999999998, 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228112+0000", 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228112+0000", 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228113+0000", 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2399999999999997e-07 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228342+0000", 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000228698 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228370+0000", 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8351e-05 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228380+0000", 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5670000000000001e-06 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228455+0000", 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5599000000000006e-05 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.228841+0000", 2024-09-17T17:30:25.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038609099999999997 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:6f166faa:::smithi10238019-38:11, version: 70'124, data_included: [0~155648], data_size: 155648, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:6f166faa:::smithi10238019-38:11@70'124, size: 622592, copy_subset: [], clone_subset: {}, snapset: 12={11: [11, 10]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 622592, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.231490+0000", 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.092523346000000006, 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060922000000000005, 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.424 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231490+0000", 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231490+0000", 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231490+0000", 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7800000000000001e-07 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231639+0000", 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000148689 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231666+0000", 2024-09-17T17:30:25.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6798999999999999e-05 2024-09-17T17:30:25.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231675+0000", 2024-09-17T17:30:25.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7180000000000008e-06 2024-09-17T17:30:25.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.231739+0000", 2024-09-17T17:30:25.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3361000000000005e-05 2024-09-17T17:30:25.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.232099+0000", 2024-09-17T17:30:25.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036007500000000001 2024-09-17T17:30:25.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.429 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:6f166faa:::smithi10238019-38:head, version: 70'125, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:6f166faa:::smithi10238019-38:head@70'125, size: 1884160, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.234809+0000", 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.089203637000000002, 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00062034700000000004, 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.234809+0000", 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.234809+0000", 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.234810+0000", 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:30:25.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.235043+0000", 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023329399999999999 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.235072+0000", 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8501e-05 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.235081+0000", 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7089999999999997e-06 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.235141+0000", 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9469000000000003e-05 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.235430+0000", 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028866599999999998 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.433 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.16s3 73/72 [PushOp(3:6f166faa:::smithi10238019-38:head, version: 70'125, data_included: [262144~208896], data_size: 208896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:6f166faa:::smithi10238019-38:head@70'125, size: 1884160, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1884160, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.237530+0000", 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.086483113, 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064869799999999996, 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237530+0000", 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237530+0000", 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237530+0000", 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5400000000000001e-07 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237711+0000", 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018087999999999999 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237737+0000", 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5922e-05 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237748+0000", 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1049e-05 2024-09-17T17:30:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.237826+0000", 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7552999999999994e-05 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.238178+0000", 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00035263999999999998 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.438 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:25.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:30:25.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:258: done (8 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:267: done (7 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:269: done (6 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:270: done (5 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:271: done (4 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:272: done (3 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:273: done (2 left) 2024-09-17T17:30:25.442 INFO:tasks.rados.rados.0.smithi102.stdout:274: done (1 left) 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:275: done (0 left) 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:276: setattr oid 30 current snap is 20 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 48 current snap is 20 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:277: seq_num 128 ranges {1884160=606208,2490368=507904,2998272=573440,3571712=475136} 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:277: writing smithi10238019-48 from 1884160 to 2490368 tid 1 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:277: writing smithi10238019-48 from 2490368 to 2998272 tid 2 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:277: writing smithi10238019-48 from 2998272 to 3571712 tid 3 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:277: writing smithi10238019-48 from 3571712 to 4046848 tid 4 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 12 current snap is 20 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:278: seq_num 129 ranges {1851392=622592,2473984=655360,3129344=688128,3817472=212992} 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:278: writing smithi10238019-12 from 1851392 to 2473984 tid 1 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:278: writing smithi10238019-12 from 2473984 to 3129344 tid 2 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:278: writing smithi10238019-12 from 3129344 to 3817472 tid 3 2024-09-17T17:30:25.443 INFO:tasks.rados.rados.0.smithi102.stdout:278: writing smithi10238019-12 from 3817472 to 4030464 tid 4 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:279: snap_create 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: finishing write tid 1 to smithi10238019-48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: finishing write tid 2 to smithi10238019-48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: finishing write tid 3 to smithi10238019-48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: finishing write tid 4 to smithi10238019-48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: finishing write tid 5 to smithi10238019-48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: finishing write tid 6 to smithi10238019-48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: oid 48 updating version 0 to 49 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: oid 48 version 49 is already newer than 48 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: oid 48 version 49 is already newer than 47 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: oid 48 version 49 is already newer than 46 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:277: oid 48 version 49 is already newer than 45 2024-09-17T17:30:25.444 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 48 v 49 (ObjNum 128 snap 20 seq_num 128) dirty exists 2024-09-17T17:30:25.445 INFO:tasks.rados.rados.0.smithi102.stdout:277: left oid 48 (ObjNum 128 snap 20 seq_num 128) 2024-09-17T17:30:25.445 INFO:tasks.rados.rados.0.smithi102.stdout:278: finishing write tid 1 to smithi10238019-12 2024-09-17T17:30:25.445 INFO:tasks.rados.rados.0.smithi102.stdout:278: finishing write tid 2 to smithi10238019-12 2024-09-17T17:30:25.445 INFO:tasks.rados.rados.0.smithi102.stdout:278: finishing write tid 3 to smithi10238019-12 2024-09-17T17:30:25.445 INFO:tasks.rados.rados.0.smithi102.stdout:278: finishing write tid 4 to smithi10238019-12 2024-09-17T17:30:25.445 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:25.438+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:25.541 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:25.539+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:25.552 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:25.553 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:25.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:25.642 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:25.640+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:25.668 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:25.668 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:25.668 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:25.668 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:25.669 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:25.676 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:25.742 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:25.741+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:25.785 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.202329767000002, 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.27275196200000001, 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580026+0000", 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.359e-06 2024-09-17T17:30:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580034+0000", 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580048+0000", 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3974e-05 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580058+0000", 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0064999999999999e-05 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580111+0000", 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3248000000000002e-05 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:30:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580119+0000", 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1270000000000003e-06 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847220+0000", 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26710079599999997 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847357+0000", 2024-09-17T17:30:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013739500000000001 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847505+0000", 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147188 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.850265+0000", 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002760396 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852701+0000", 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024356059999999999 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852744+0000", 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3095e-05 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852777+0000", 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2784000000000003e-05 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794958788000002, 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010915796, 2024-09-17T17:30:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 46 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987396+0000", 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5400000000000002e-07 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987400+0000", 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1450000000000001e-06 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987405+0000", 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2739999999999998e-06 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987409+0000", 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1370000000000004e-06 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990238+0000", 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028288110000000001 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990293+0000", 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4657999999999997e-05 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990394+0000", 2024-09-17T17:30:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010156 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991880+0000", 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014857169999999999 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998207+0000", 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0063267899999999997 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998262+0000", 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998311+0000", 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9777999999999997e-05 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794925673000002, 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010963481000000001, 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 47 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987429+0000", 2024-09-17T17:30:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5400000000000001e-07 2024-09-17T17:30:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987478+0000", 2024-09-17T17:30:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9118e-05 2024-09-17T17:30:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987490+0000", 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1425e-05 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987502+0000", 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2354999999999999e-05 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990908+0000", 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003406002 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.990958+0000", 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9597e-05 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991063+0000", 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010533 2024-09-17T17:30:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993063+0000", 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019997909999999999 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998320+0000", 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052566959999999999 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998349+0000", 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9200000000000002e-05 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998392+0000", 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3412999999999999e-05 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794841478999999, 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010965536999999999, 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 48 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5899999999999996e-07 2024-09-17T17:30:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987518+0000", 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0429999999999998e-06 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987523+0000", 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3499999999999999e-06 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987526+0000", 2024-09-17T17:30:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5159999999999999e-06 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991339+0000", 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038128670000000002 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991396+0000", 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7219000000000002e-05 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991498+0000", 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010142 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993114+0000", 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016166900000000001 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998400+0000", 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052850600000000003 2024-09-17T17:30:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998438+0000", 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8272000000000002e-05 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998478+0000", 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0540999999999997e-05 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794823129000001, 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011028196000000001, 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 49 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987534+0000", 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4399999999999999e-06 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987538+0000", 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9369999999999997e-06 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987541+0000", 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0350000000000002e-06 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991920+0000", 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043794000000000003 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991972+0000", 2024-09-17T17:30:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1412000000000002e-05 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992057+0000", 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4917e-05 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993157+0000", 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011003810000000001 2024-09-17T17:30:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998485+0000", 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053282989999999999 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998517+0000", 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2019e-05 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998559+0000", 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2076000000000002e-05 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794801640000003, 2024-09-17T17:30:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011080325, 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 50 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-07 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987605+0000", 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2114000000000003e-05 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987631+0000", 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5766999999999999e-05 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987646+0000", 2024-09-17T17:30:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515999999999998e-05 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992371+0000", 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047247360000000002 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992433+0000", 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1451000000000005e-05 2024-09-17T17:30:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992522+0000", 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9621000000000001e-05 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994510+0000", 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001988227 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998564+0000", 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040533710000000001 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998595+0000", 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0843000000000003e-05 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998633+0000", 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8321000000000001e-05 2024-09-17T17:30:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794689615000003, 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011043702000000001, 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 51 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987669+0000", 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7189999999999999e-06 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987674+0000", 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1780000000000002e-06 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987678+0000", 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5999999999999998e-06 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993192+0000", 2024-09-17T17:30:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055137620000000002 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993262+0000", 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993349+0000", 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7362999999999997e-05 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994549+0000", 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011994110000000001 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998640+0000", 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040912409999999998 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998669+0000", 2024-09-17T17:30:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9209000000000001e-05 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998708+0000", 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9419000000000003e-05 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.794669525000003, 2024-09-17T17:30:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011098023, 2024-09-17T17:30:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 52 2024-09-17T17:30:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2000000000000001e-07 2024-09-17T17:30:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987688+0000", 2024-09-17T17:30:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4729999999999999e-06 2024-09-17T17:30:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987693+0000", 2024-09-17T17:30:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5010000000000004e-06 2024-09-17T17:30:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987698+0000", 2024-09-17T17:30:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1499999999999998e-06 2024-09-17T17:30:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994130+0000", 2024-09-17T17:30:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064314890000000003 2024-09-17T17:30:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994185+0000", 2024-09-17T17:30:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5769000000000001e-05 2024-09-17T17:30:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994272+0000", 2024-09-17T17:30:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6638000000000006e-05 2024-09-17T17:30:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994990+0000", 2024-09-17T17:30:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000717463 2024-09-17T17:30:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998715+0000", 2024-09-17T17:30:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003725 2024-09-17T17:30:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998743+0000", 2024-09-17T17:30:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8830999999999999e-05 2024-09-17T17:30:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998783+0000", 2024-09-17T17:30:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9388999999999998e-05 2024-09-17T17:30:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.484955030999998, 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011397086000000001, 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 76 2024-09-17T17:30:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:30:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297400+0000", 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7100000000000002e-07 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297403+0000", 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5920000000000001e-06 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297410+0000", 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1489999999999997e-06 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297417+0000", 2024-09-17T17:30:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-06 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306131+0000", 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087138279999999999 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306185+0000", 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3783999999999999e-05 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306278+0000", 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3587999999999999e-05 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306994+0000", 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071571099999999995 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308729+0000", 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001735489 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308769+0000", 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9270000000000002e-05 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308796+0000", 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7603999999999999e-05 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:124 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.242893479000003, 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011177351, 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 124 2024-09-17T17:30:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539467+0000", 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2089999999999996e-06 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539473+0000", 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4690000000000002e-06 2024-09-17T17:30:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539479+0000", 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545059+0000", 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055806400000000004 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545104+0000", 2024-09-17T17:30:25.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5040999999999998e-05 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545172+0000", 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7249999999999995e-05 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545874+0000", 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070271799999999996 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550512+0000", 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046379530000000002 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550597+0000", 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025999999999995e-05 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550638+0000", 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0771e-05 2024-09-17T17:30:25.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.as2 37/32 ECSubRead(tid=27, to_read={3:5ee91470:::smithi10238019-6:head=20480,4096,0}, subchunks={3:5ee91470:::smithi10238019-6:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.789327727, 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.671263006, 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.868 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993028+0000", 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0219999999999999e-06 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993040+0000", 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1968e-05 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664125+0000", 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.67108543799999998 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664134+0000", 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8640000000000002e-06 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664204+0000", 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9733999999999993e-05 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664290+0000", 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.598e-05 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.871 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 43/35 ECSubWrite(tid=54, reqid=osd.11.0:54, at_version=43'10, trim_to=0'0, roll_forward_to=42'8))", 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.080835775000001, 2024-09-17T17:30:25.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.282394969, 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:25.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6e-07 2024-09-17T17:30:25.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701543+0000", 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4530000000000001e-05 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983000+0000", 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.28145619700000002 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983008+0000", 2024-09-17T17:30:25.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6510000000000007e-06 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983087+0000", 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8623999999999998e-05 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983093+0000", 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6599999999999998e-06 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983867+0000", 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000773962 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983914+0000", 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6084999999999997e-05 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.878 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:301 3.8s0 3:11ea5ca6:::smithi10238019-33:head [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.746448+0000", 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.035905859, 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0099540300000000009, 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 301 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.746448+0000", 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.746448+0000", 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.746449+0000", 2024-09-17T17:30:25.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.746454+0000", 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4850000000000003e-06 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.746461+0000", 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7039999999999994e-06 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.746467+0000", 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.747982+0000", 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015146479999999999 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.748053+0000", 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1057999999999996e-05 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.756402+0000", 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0083498570000000005 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 43/35 ECSubWrite(tid=62, reqid=client.4566.0:316, at_version=43'14, trim_to=0'0, roll_forward_to=43'10))", 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.791388823999998, 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.039163349, 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990966+0000", 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3e-07 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990998+0000", 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1909000000000002e-05 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029506+0000", 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.038507953999999997 2024-09-17T17:30:25.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029512+0000", 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3419999999999996e-06 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029561+0000", 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8930000000000001e-05 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029568+0000", 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2480000000000001e-06 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.030089+0000", 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00052149900000000003 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.030129+0000", 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9737000000000003e-05 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:424 3.8s0 3:1f25649f:::smithi10238019-40:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.382178+0000", 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.400176049999999, 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0094149400000000001, 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:25.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 424 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382178+0000", 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382178+0000", 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382179+0000", 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.37e-07 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382183+0000", 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0840000000000002e-06 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382191+0000", 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6199999999999999e-06 2024-09-17T17:30:25.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382197+0000", 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.947e-06 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.384314+0000", 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021175619999999999 2024-09-17T17:30:25.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.384430+0000", 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011582300000000001 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.391593+0000", 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0071632670000000001 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.401922859999999, 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31954210500000002, 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380432+0000", 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8500000000000001e-07 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380446+0000", 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4214e-05 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699334+0000", 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31888741599999998 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699350+0000", 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5913000000000002e-05 2024-09-17T17:30:25.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699521+0000", 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000171244 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699530+0000", 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8079999999999994e-06 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:25.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699939+0000", 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040906299999999998 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699974+0000", 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4761999999999997e-05 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 68/67 ECSubWrite(tid=123, reqid=client.4566.0:616, at_version=0'0, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:25.897 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.449778+0000", 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.3325760139999998, 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01031115, 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449778+0000", 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449778+0000", 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449779+0000", 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449791+0000", 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1671000000000001e-05 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.459323+0000", 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095322549999999999 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.459331+0000", 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6879999999999999e-06 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.459401+0000", 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0487000000000005e-05 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.459415+0000", 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3859000000000001e-05 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.460046+0000", 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00063100700000000005 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.460089+0000", 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3559000000000001e-05 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.902 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.18s0 68/67 ECSubWriteReply(tid=153, last_complete=68'25, committed=1, applied=1))", 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.318425672, 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.73874762000000005, 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0400000000000002e-07 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463933+0000", 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9689999999999996e-06 2024-09-17T17:30:25.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202459+0000", 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.738526142 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202472+0000", 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2989999999999999e-05 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202624+0000", 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015171499999999999 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202676+0000", 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.24e-05 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.906 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 68/67 ECSubWrite(tid=125, reqid=client.4566.0:619, at_version=68'49, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.466068+0000", 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.316286174, 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010361113, 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466068+0000", 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466068+0000", 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466069+0000", 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466296+0000", 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000227452 2024-09-17T17:30:25.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475369+0000", 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090730340000000007 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475378+0000", 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9989999999999997e-06 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475441+0000", 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2237999999999996e-05 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475450+0000", 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9660000000000002e-06 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476376+0000", 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092614800000000001 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476429+0000", 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3458999999999998e-05 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.3063641119999998, 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.37910365400000001, 2024-09-17T17:30:25.911 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:25.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475991+0000", 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476005+0000", 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.449e-05 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:25.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854381+0000", 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.37837595699999998 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854387+0000", 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4689999999999996e-06 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854441+0000", 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4593999999999999e-05 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854449+0000", 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0029999999999994e-06 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.855065+0000", 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061543700000000004 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.855094+0000", 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9193999999999999e-05 2024-09-17T17:30:25.915 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.916 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:25.916 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:25.916 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:25.917 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:25.916+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:25.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:26.018 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:26.016+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:26.026 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:26.026 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:26.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.026 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:725 3.3s0 3.184a5fa3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.68714840600000004, 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68716512299999999, 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "waiting for peered", 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 725 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337666+0000", 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.513e-06 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337795+0000", 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000128961 2024-09-17T17:30:26.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337808+0000", 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3713000000000001e-05 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337819+0000", 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.027e-05 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341249+0000", 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034306190000000002 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341253+0000", 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4410000000000002e-06 2024-09-17T17:30:26.031 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.032 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.032 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.032 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.032 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-17T17:30:26.032 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-17T17:30:26.032 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:26.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:26.119 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:26.118+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:26.141 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:26.141 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:26.141 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:26.141 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:26.141 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:26.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:26.220 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:26.219+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:26.258 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:26.258 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:26.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:186 3.3s0 3:c77cc1f5:::smithi10238019-3:head [append 0~0] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.677940+0000", 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.577992064999997, 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0063471229999999997, 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 186 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.677940+0000", 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.677940+0000", 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.677940+0000", 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4200000000000001e-07 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.677944+0000", 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9280000000000003e-06 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.677951+0000", 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3289999999999999e-06 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.677955+0000", 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.809e-06 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.679666+0000", 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0017110300000000001 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.679751+0000", 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4167999999999998e-05 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.680121+0000", 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000369983 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.680461+0000", 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00034078899999999998 2024-09-17T17:30:26.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.684214+0000", 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037527010000000002 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.684287+0000", 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2943999999999998e-05 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:218 3.3s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.534317121000001, 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65573700899999998, 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 218 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721616+0000", 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4e-07 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721619+0000", 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2030000000000001e-06 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721626+0000", 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:26.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721631+0000", 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4539999999999998e-06 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722624+0000", 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00099297500000000011 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722675+0000", 2024-09-17T17:30:26.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0627000000000002e-05 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722943+0000", 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026790700000000001 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723463+0000", 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000520003 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377270+0000", 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65380684099999997 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377352+0000", 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2307999999999999e-05 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:219 3.3s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.534290896999998, 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65649164999999998, 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 219 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.271 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721642+0000", 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0100000000000005e-07 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721644+0000", 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4719999999999998e-06 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721651+0000", 2024-09-17T17:30:26.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.781e-06 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721656+0000", 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4809999999999997e-06 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723147+0000", 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001490587 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723197+0000", 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.003e-05 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377653+0000", 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65445650300000002 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377716+0000", 2024-09-17T17:30:26.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2223000000000004e-05 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.378133+0000", 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041707199999999999 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:439 3.3s0 3:c80ea1ca:::smithi10238019-27:head [append 0~688128] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.403716+0000", 2024-09-17T17:30:26.276 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.852215761, 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.007734533, 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 439 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.403716+0000", 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.403716+0000", 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.403717+0000", 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0210000000000001e-06 2024-09-17T17:30:26.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404347+0000", 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00062963799999999996 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404357+0000", 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0062e-05 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404363+0000", 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0750000000000003e-06 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404531+0000", 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016821499999999999 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404594+0000", 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2491999999999997e-05 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.405401+0000", 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000806863 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.408110+0000", 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002709099 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.411368+0000", 2024-09-17T17:30:26.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00325845 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.411451+0000", 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2618000000000001e-05 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:440 3.3s0 3:c80ea1ca:::smithi10238019-27:head [append 0~81920] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:26.282 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.851557374999999, 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19940718800000001, 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 440 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8299999999999996e-07 2024-09-17T17:30:26.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404418+0000", 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2608000000000003e-05 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404437+0000", 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8810000000000001e-05 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404442+0000", 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.327e-06 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406246+0000", 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018034769999999999 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406352+0000", 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106269 2024-09-17T17:30:26.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406784+0000", 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00043238800000000002 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.408111+0000", 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001326502 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603719+0000", 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.195608474 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603782+0000", 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2550000000000003e-05 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/35 ECSubWriteReply(tid=48, last_complete=52'36, committed=1, applied=1))", 2024-09-17T17:30:26.288 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.848162237, 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19602166600000001, 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407771+0000", 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1100000000000004e-07 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407775+0000", 2024-09-17T17:30:26.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5510000000000001e-06 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603542+0000", 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19576712299999999 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603557+0000", 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4229999999999999e-05 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603658+0000", 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010186100000000001 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603791+0000", 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013299000000000001 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.292 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.2s5 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.338063+0000", 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.917868562999999, 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060186179999999999, 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338063+0000", 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338063+0000", 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338064+0000", 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2500000000000001e-07 2024-09-17T17:30:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338098+0000", 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4091000000000003e-05 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338186+0000", 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7647999999999998e-05 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338196+0000", 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0621e-05 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338349+0000", 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000153096 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.338361+0000", 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1415e-05 2024-09-17T17:30:26.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.344043+0000", 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0056826289999999998 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.344082+0000", 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8792999999999999e-05 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.as3 57/32 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=57'63, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.372715+0000", 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.883217319, 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0070603990000000002, 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372715+0000", 2024-09-17T17:30:26.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372715+0000", 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372715+0000", 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2500000000000005e-07 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372733+0000", 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7679000000000001e-05 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372776+0000", 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2586999999999997e-05 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372786+0000", 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0386e-05 2024-09-17T17:30:26.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372866+0000", 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9624999999999997e-05 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.372879+0000", 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3388e-05 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.379728+0000", 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068488949999999998 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.379775+0000", 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7114000000000003e-05 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s3 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.376293+0000", 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.879638811, 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006275213, 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376293+0000", 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376293+0000", 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376293+0000", 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.17e-07 2024-09-17T17:30:26.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376307+0000", 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3291e-05 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376370+0000", 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3689000000000006e-05 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376377+0000", 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2079999999999997e-06 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376436+0000", 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9741000000000003e-05 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.376444+0000", 2024-09-17T17:30:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4259999999999997e-06 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.382501+0000", 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060577720000000003 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.382568+0000", 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6969000000000002e-05 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s3 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.378371+0000", 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.877561274000001, 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0066834010000000003, 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378371+0000", 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378371+0000", 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378372+0000", 2024-09-17T17:30:26.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9999999999999996e-07 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378391+0000", 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9868999999999999e-05 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378433+0000", 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1916999999999999e-05 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378440+0000", 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7020000000000004e-06 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378495+0000", 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4500999999999999e-05 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378501+0000", 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0780000000000001e-06 2024-09-17T17:30:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.385008+0000", 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0065076300000000004 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.385054+0000", 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5904e-05 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.as3 62/32 ECSubWrite(tid=104, reqid=client.4566.0:558, at_version=62'84, trim_to=0'0, roll_forward_to=62'79))", 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.753997203000001, 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95386532199999996, 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501936+0000", 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0109999999999999e-06 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501948+0000", 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2544000000000001e-05 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454818+0000", 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95286919199999998 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454827+0000", 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3130000000000004e-06 2024-09-17T17:30:26.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454959+0000", 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013207099999999999 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454969+0000", 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0368e-05 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455761+0000", 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000792051 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455800+0000", 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8772000000000001e-05 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=123, reqid=client.4566.0:616, at_version=0'0, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.8061196470000001, 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015260055, 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449813+0000", 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449823+0000", 2024-09-17T17:30:26.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9539999999999999e-06 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464497+0000", 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014673699 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464503+0000", 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1770000000000003e-06 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464557+0000", 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464568+0000", 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0386e-05 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465032+0000", 2024-09-17T17:30:26.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046443199999999997 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465072+0000", 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0280000000000001e-05 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=124, reqid=client.4566.0:618, at_version=68'48, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.464025+0000", 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.7919070619999999, 2024-09-17T17:30:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.007291596, 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464025+0000", 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464025+0000", 2024-09-17T17:30:26.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464026+0000", 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9999999999999997e-07 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464480+0000", 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000454563 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466657+0000", 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002177277 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466665+0000", 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5959999999999998e-06 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466724+0000", 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8947999999999999e-05 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466734+0000", 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9320000000000008e-06 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.471278+0000", 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045437289999999998 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.471317+0000", 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8850999999999998e-05 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=125, reqid=client.4566.0:619, at_version=68'49, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.7895962240000003, 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010346985, 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466642+0000", 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030544600000000002 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475441+0000", 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087995179999999992 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475448+0000", 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4439999999999996e-06 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475494+0000", 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000003e-05 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475500+0000", 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476652+0000", 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001152887 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476683+0000", 2024-09-17T17:30:26.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0306000000000001e-05 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.7798601239999998, 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86543154899999997, 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.332 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476073+0000", 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4900000000000005e-07 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476093+0000", 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0431000000000001e-05 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340542+0000", 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86444875899999996 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340554+0000", 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2017e-05 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340635+0000", 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1500000000000002e-05 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340651+0000", 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5444999999999999e-05 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341467+0000", 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00081634899999999996 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341503+0000", 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6298999999999999e-05 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:630 3.3s0 3:c77cc1f5:::smithi10238019-3:head [read 0~2686976,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.500497+0000", 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.7554351769999998, 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060379120000000003, 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 630 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500497+0000", 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500497+0000", 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500499+0000", 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0990000000000001e-06 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500508+0000", 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8300000000000002e-06 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500532+0000", 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3822e-05 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500546+0000", 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4778e-05 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500622+0000", 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5907000000000004e-05 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500783+0000", 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016116399999999999 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.506535+0000", 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057513119999999997 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:631 3.3s0 3:c77cc1f5:::smithi10238019-3:head [read 954514~1732462,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.500573+0000", 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.7553589970000001, 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069449639999999997, 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 631 2024-09-17T17:30:26.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500573+0000", 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500573+0000", 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500574+0000", 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8699999999999995e-07 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500578+0000", 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3290000000000004e-06 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500586+0000", 2024-09-17T17:30:26.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500594+0000", 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3990000000000004e-06 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.501188+0000", 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059365600000000002 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.501269+0000", 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1128e-05 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.507518+0000", 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0062487039999999999 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:632 3.3s0 3:c77cc1f5:::smithi10238019-3:head [read 2600511~86465,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.500604+0000", 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.7553283759999996, 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074507740000000003, 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 632 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500604+0000", 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500604+0000", 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500604+0000", 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1699999999999998e-07 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500608+0000", 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6399999999999999e-06 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500619+0000", 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.151e-05 2024-09-17T17:30:26.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.500626+0000", 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2210000000000002e-06 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.503564+0000", 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002938567 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.503661+0000", 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7366000000000003e-05 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.508054+0000", 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043929529999999998 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.351 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:644 3.3s0 3:c77cc1f5:::smithi10238019-3:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.556838484, 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.524887887, 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 644 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.352 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699095+0000", 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.099e-06 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699100+0000", 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1549999999999998e-06 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699109+0000", 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1670000000000005e-06 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699117+0000", 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8720000000000002e-06 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699171+0000", 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4237999999999998e-05 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699278+0000", 2024-09-17T17:30:26.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107339 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699941+0000", 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066276400000000004 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.702383+0000", 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024413550000000001 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223916+0000", 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52153379200000005 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223981+0000", 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5105999999999996e-05 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 69/67 ECSubWrite(tid=129, reqid=client.4566.0:621, at_version=0'0, trim_to=0'0, roll_forward_to=68'51))", 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.9131441139999996, 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014148114, 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342789+0000", 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.13e-06 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342803+0000", 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4309e-05 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356208+0000", 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013404337000000001 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356219+0000", 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1569000000000001e-05 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356293+0000", 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3894999999999996e-05 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356301+0000", 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5159999999999997e-06 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356892+0000", 2024-09-17T17:30:26.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059179499999999999 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356936+0000", 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3563000000000003e-05 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.362 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:30:26.364 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:26.362+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:26.365 INFO:tasks.rados.rados.0.smithi102.stdout:278: finishing write tid 5 to smithi10238019-12 2024-09-17T17:30:26.365 INFO:tasks.rados.rados.0.smithi102.stdout:278: finishing write tid 6 to smithi10238019-12 2024-09-17T17:30:26.365 INFO:tasks.rados.rados.0.smithi102.stdout:278: oid 12 updating version 0 to 114 2024-09-17T17:30:26.365 INFO:tasks.rados.rados.0.smithi102.stdout:278: oid 12 version 114 is already newer than 110 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:278: oid 12 version 114 is already newer than 113 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:278: oid 12 version 114 is already newer than 111 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:278: oid 12 version 114 is already newer than 112 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 12 v 114 (ObjNum 129 snap 20 seq_num 129) dirty exists 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:278: left oid 12 (ObjNum 129 snap 20 seq_num 129) 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:277: done (3 left) 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:278: done (2 left) 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:279: done (1 left) 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:280: snap_create 2024-09-17T17:30:26.366 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 30 v 56 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-09-17T17:30:26.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:26.464 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:26.463+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:26.475 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:26.475 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:26.475 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:26.475 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:26.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:26.565 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:26.563+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:26.631 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:26.631 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:26.631 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:26.631 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:26.631 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:26.639 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:26.666 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:26.664+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:26.748 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:308 3.2s0 3:4c833192:::smithi10238019-39:head [rollback 5] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.775718995999998, 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076693619999999999, 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 308 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.68e-07 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970369+0000", 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7160000000000002e-06 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970378+0000", 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5739999999999996e-06 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970383+0000", 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0010000000000001e-06 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971568+0000", 2024-09-17T17:30:26.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011857020000000001 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971627+0000", 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8826999999999998e-05 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972103+0000", 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047574100000000001 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973681+0000", 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001578404 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.977986+0000", 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043046630000000002 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978033+0000", 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7166000000000002e-05 2024-09-17T17:30:26.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:309 3.2s0 3:4c833192:::smithi10238019-39:head [append 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.775687953999999, 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077796990000000002, 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 309 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:26.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2e-07 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970399+0000", 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0779999999999997e-06 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970408+0000", 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7979999999999995e-06 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970415+0000", 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9310000000000002e-06 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972569+0000", 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021534969999999999 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972677+0000", 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108145 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973250+0000", 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057328100000000003 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973923+0000", 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00067244899999999999 2024-09-17T17:30:26.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978120+0000", 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041977949999999998 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978175+0000", 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4205e-05 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:326 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698785+0000", 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.047298283, 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068833499999999999, 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 326 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698785+0000", 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698785+0000", 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698787+0000", 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7209999999999999e-06 2024-09-17T17:30:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698798+0000", 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1032999999999999e-05 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698819+0000", 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1005999999999998e-05 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698830+0000", 2024-09-17T17:30:26.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0858e-05 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698902+0000", 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1996000000000007e-05 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698911+0000", 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1859999999999998e-06 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.701053+0000", 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002141479 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.701152+0000", 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9332000000000004e-05 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702018+0000", 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086619400000000004 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704464+0000", 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024453949999999999 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.705564+0000", 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001100668 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.705668+0000", 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103482 2024-09-17T17:30:26.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:327 3.2s0 3:497089da:::smithi10238019-9:head [rollback 5] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.047233533, 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0072194929999999996, 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 327 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698850+0000", 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698852+0000", 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3939999999999999e-06 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698858+0000", 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1480000000000003e-06 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698863+0000", 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3470000000000001e-06 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698988+0000", 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012491999999999999 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698997+0000", 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6819999999999998e-06 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702506+0000", 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035088879999999999 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702618+0000", 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111428 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703259+0000", 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064172000000000003 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704549+0000", 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001289599 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706021+0000", 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001471857 2024-09-17T17:30:26.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706069+0000", 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8072000000000003e-05 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:328 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.047193991, 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0073597310000000004, 2024-09-17T17:30:26.773 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 328 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7e-07 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698894+0000", 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.057e-06 2024-09-17T17:30:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698902+0000", 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3760000000000002e-06 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698907+0000", 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0329999999999999e-06 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699035+0000", 2024-09-17T17:30:26.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012849700000000001 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699042+0000", 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.55e-06 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703631+0000", 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004589536 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703735+0000", 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103891 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704152+0000", 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041683699999999998 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704821+0000", 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066855199999999999 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706206+0000", 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001385788 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706249+0000", 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2243999999999997e-05 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 52/35 ECSubWrite(tid=125, reqid=osd.13.0:125, at_version=52'57, trim_to=0'0, roll_forward_to=51'53))", 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.402002027999998, 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0071485719999999997, 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344082+0000", 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.85e-07 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344108+0000", 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6270000000000001e-05 2024-09-17T17:30:26.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344147+0000", 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9462999999999998e-05 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344158+0000", 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0786e-05 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344212+0000", 2024-09-17T17:30:26.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4143e-05 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344220+0000", 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8390000000000007e-06 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.351178+0000", 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006957876 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.351229+0000", 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1409999999999997e-05 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:421 3.2s0 3:4c833192:::smithi10238019-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.363441767000001, 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090389679999999997, 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 421 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382642+0000", 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000005e-07 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382644+0000", 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382652+0000", 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5059999999999998e-06 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382656+0000", 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8029999999999996e-06 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385875+0000", 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0032189900000000001 2024-09-17T17:30:26.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385954+0000", 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9087999999999999e-05 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388823+0000", 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028687090000000001 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388909+0000", 2024-09-17T17:30:26.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5457000000000005e-05 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.391680+0000", 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027713939999999999 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 53/35 ECSubWrite(tid=133, reqid=osd.13.0:133, at_version=53'64, trim_to=0'0, roll_forward_to=52'62))", 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:26.789 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.384038581999999, 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078716470000000007, 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:26.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362068+0000", 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3370000000000002e-05 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362467+0000", 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039876899999999999 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362476+0000", 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8740000000000001e-06 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364044+0000", 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001568261 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364055+0000", 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.077e-05 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369861+0000", 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0058066660000000003 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369916+0000", 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4691999999999997e-05 2024-09-17T17:30:26.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s5 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.406665443000001, 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074908880000000002, 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339418+0000", 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:26.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339449+0000", 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.133e-05 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339554+0000", 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104682 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339563+0000", 2024-09-17T17:30:26.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9129999999999992e-06 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339937+0000", 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037396800000000002 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339947+0000", 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0451999999999999e-05 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346872+0000", 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069250020000000004 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346908+0000", 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6117e-05 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs3 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.339535+0000", 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.406548175000001, 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068455240000000004, 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339535+0000", 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339535+0000", 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339536+0000", 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0800000000000003e-07 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339563+0000", 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6982999999999999e-05 2024-09-17T17:30:26.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339600+0000", 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7599999999999999e-05 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339608+0000", 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1499999999999999e-06 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339660+0000", 2024-09-17T17:30:26.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1758999999999997e-05 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339680+0000", 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0173000000000002e-05 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346338+0000", 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006658181 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346380+0000", 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1770000000000002e-05 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s2 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.378147+0000", 2024-09-17T17:30:26.803 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.367935639999999, 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067666209999999996, 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378147+0000", 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378147+0000", 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378148+0000", 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.279e-06 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378184+0000", 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5629000000000001e-05 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378240+0000", 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5699999999999999e-05 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378249+0000", 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.054e-06 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378313+0000", 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4425999999999996e-05 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.378321+0000", 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6490000000000008e-06 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.384869+0000", 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0065477260000000002 2024-09-17T17:30:26.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.384914+0000", 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5157999999999997e-05 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=154, reqid=osd.13.0:154, at_version=57'84, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.356167971000001, 2024-09-17T17:30:26.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035993311, 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389916+0000", 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0299999999999997e-07 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389936+0000", 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0061e-05 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425016+0000", 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035080622999999998 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425025+0000", 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3110000000000006e-06 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425088+0000", 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3442999999999995e-05 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425103+0000", 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4625e-05 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425875+0000", 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000772585 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425908+0000", 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2759999999999998e-05 2024-09-17T17:30:26.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=155, reqid=osd.13.0:155, at_version=57'86, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.355266791999998, 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26589276499999998, 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390817+0000", 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-17T17:30:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390839+0000", 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2725999999999999e-05 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655507+0000", 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26466748600000001 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655515+0000", 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5530000000000004e-06 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655625+0000", 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109997 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655635+0000", 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0000000000000001e-05 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656673+0000", 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001038464 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656709+0000", 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5830000000000001e-05 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=156, reqid=client.4566.0:492, at_version=57'10, trim_to=0'0, roll_forward_to=43'8))", 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.321106317000002, 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23215693400000001, 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4900000000000004e-07 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424998+0000", 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0800000000000001e-05 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655688+0000", 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23068955499999999 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655696+0000", 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0839999999999992e-06 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655818+0000", 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012232399999999999 2024-09-17T17:30:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655830+0000", 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1355000000000001e-05 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657105+0000", 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001275403 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657133+0000", 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8464000000000001e-05 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=158, reqid=client.4566.0:492, at_version=0'0, trim_to=0'0, roll_forward_to=57'10))", 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.088261253999999, 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26690073399999997, 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000001e-07 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657834+0000", 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1450999999999999e-05 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923785+0000", 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26595101999999998 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923800+0000", 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.508e-05 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923956+0000", 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155876 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923975+0000", 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9443e-05 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924688+0000", 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071237099999999995 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924722+0000", 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4629000000000003e-05 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 59/35 ECSubWrite(tid=164, reqid=client.4566.0:513, at_version=59'90, trim_to=0'0, roll_forward_to=59'88))", 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.370540576, 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080861479999999996, 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375543+0000", 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4499999999999995e-07 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375918+0000", 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037515399999999999 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375961+0000", 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2206000000000002e-05 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375969+0000", 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8349999999999993e-06 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376050+0000", 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0269e-05 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376060+0000", 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0509999999999999e-05 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383584+0000", 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075235939999999998 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383628+0000", 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4635000000000002e-05 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 59/35 ECSubWrite(tid=165, reqid=client.4566.0:514, at_version=59'91, trim_to=0'0, roll_forward_to=59'88))", 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:10.376940+0000", 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.369142848999999, 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067160639999999999, 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376940+0000", 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376940+0000", 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376941+0000", 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376966+0000", 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5289000000000001e-05 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376998+0000", 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1542999999999999e-05 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.377004+0000", 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3439999999999997e-06 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.377075+0000", 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0889000000000005e-05 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.377085+0000", 2024-09-17T17:30:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0509999999999999e-05 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383635+0000", 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0065491600000000001 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383656+0000", 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1535e-05 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:562 3.2s0 3:497089da:::smithi10238019-9:head [rollback f] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.329949245, 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076874359999999997, 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 562 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1299999999999999e-07 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416138+0000", 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4920000000000002e-06 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416146+0000", 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8800000000000008e-06 2024-09-17T17:30:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416150+0000", 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6840000000000004e-06 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416271+0000", 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012026799999999999 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416278+0000", 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5889999999999997e-06 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419622+0000", 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0033434419999999999 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419740+0000", 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011798600000000001 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420292+0000", 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00055223900000000005 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421629+0000", 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001337429 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423776+0000", 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021470370000000001 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423821+0000", 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4676999999999999e-05 2024-09-17T17:30:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:563 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.329925393, 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079319400000000002, 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 563 2024-09-17T17:30:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416158+0000", 2024-09-17T17:30:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416161+0000", 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6649999999999999e-06 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416171+0000", 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0176e-05 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416181+0000", 2024-09-17T17:30:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0327e-05 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416310+0000", 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129247 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416317+0000", 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0720000000000002e-06 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420645+0000", 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043279629999999998 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420731+0000", 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5937000000000001e-05 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421175+0000", 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044333000000000003 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423834+0000", 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002659121 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424046+0000", 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021196999999999999 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424089+0000", 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3702999999999998e-05 2024-09-17T17:30:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s1 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.363874+0000", 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.382209311, 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006768937, 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363874+0000", 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363874+0000", 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363874+0000", 2024-09-17T17:30:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363888+0000", 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363911+0000", 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3459999999999999e-05 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363919+0000", 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2869999999999999e-06 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363959+0000", 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0296000000000002e-05 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363964+0000", 2024-09-17T17:30:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3279999999999996e-06 2024-09-17T17:30:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370613+0000", 2024-09-17T17:30:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0066489280000000001 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370642+0000", 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9132000000000002e-05 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:26.854 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:26.856 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:26.856 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:26.854+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:26.957 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:26.955+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:26.965 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:26.965 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:26.965 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:26.965 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:26.974 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:27.061 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:27.059+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:27.117 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:27.117 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:27.117 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:27.117 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:27.117 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:27.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:27.125 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:27.162 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:27.160+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:169 3.as0 3:5ee91470:::smithi10238019-6:head [read 83682~14622,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.236885313000002, 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67448544799999999, 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.230 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 169 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.231 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990278+0000", 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6400000000000002e-07 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990283+0000", 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3900000000000001e-06 2024-09-17T17:30:27.232 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990291+0000", 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9379999999999992e-06 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990297+0000", 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6099999999999997e-06 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.233 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992308+0000", 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0020111650000000001 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992380+0000", 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1459999999999997e-05 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.664763+0000", 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67238332099999998 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.234 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 37/32 ECSubWriteReply(tid=24, last_complete=37'12, committed=1, applied=1))", 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.226853470999998, 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069166769000000003, 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.235 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000310+0000", 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1299999999999998e-07 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.236 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000314+0000", 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.461e-06 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069373+0000", 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069059000999999995 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069380+0000", 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2570000000000001e-06 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.237 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069437+0000", 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7519999999999998e-05 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069476+0000", 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8617e-05 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.238 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.157819154999999, 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52672497699999998, 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.239 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069345+0000", 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.048e-06 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069355+0000", 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0052e-05 2024-09-17T17:30:27.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595186+0000", 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52583139400000001 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595201+0000", 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4790000000000001e-05 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595348+0000", 2024-09-17T17:30:27.241 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014743699999999999 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595363+0000", 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4589999999999999e-05 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596033+0000", 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067039499999999995 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.242 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596069+0000", 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5271000000000002e-05 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:223 3.as0 3:5a070e56:::smithi10238019-47:head [read 351913~434519,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 36.501149863999999, 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010342224000000001, 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:27.243 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 223 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.244 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7299999999999996e-07 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726018+0000", 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0770000000000001e-06 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.245 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726027+0000", 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7210000000000006e-06 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726034+0000", 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3419999999999996e-06 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.728268+0000", 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002233886 2024-09-17T17:30:27.246 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.728356+0000", 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8360000000000001e-05 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.736355+0000", 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079992650000000002 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.247 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:390 3.as0 3:5ee91470:::smithi10238019-6:head [create,setxattr `sjUtrS\\bejkdS^PZ^UbhfeR]sl^p__anSUR_aom (1578) in=40b,setxattr Xgg (981) in=3b,setxattr jjU_`S[ao\\\\^aUS`bZo^cefhdeaegSZfc]sS[m[k (631) in=40b,setxattr WptVXs^]ZljihqWVjRTTRsYbpoOd[kede^[d^iNS (15648) in=40b,setxattr NUPa_VdcRWiTY (981) in=13b,setxattr oapqrhbaXndiek^ec[NZPnlNjeX^`^^ht`__ll[g (630) in=40b,setxattr hUokalb_dNqZaP[SZdSptRb]ZndqQTkPSdWpXlg_ (1683) in=40b,setxattr SOaUaOtoOmejUfsqfadVc_\\fpWak\\^nknRQfaXOZ (9120) in=40b,setxattr r[RO_Pl (981) in=7b,setxattr OOd^ZXYi[]NO^mlSkQafkPlVaY`UYcd\\bnaQUZjq (1225) in=40b,setxattr mlY]_qqgYNhaYetZtkgQPN^a^VmW[amS`Qbadgjr (137) in=40b,setxattr iVYdjchT[j\\[\\\\Uto`qnjUcXas^PtqZa^VoRktTT (1823) in=40b,setxattr PhV`]ZkggalnUas (981) in=15b,setxattr [TNbUtYOkT`ZatPerOQOlk\\WahmgadNWaeRZ^YT\\ (1121) in=40b,setxattr aRqq]f]b_cobaZ`ZZRaattjja[X``PaT\\l\\^Y[jU (313) in=40b,setxattr qYXjVRWg (981) in=8b,setxattr cm`nfp^lmRcSZUsStaOg_omiONqeTRWUsOjm]png (1227) in=40b,setxattr _jtbmRhQeZe]nY\\TQctPVagWm[isibp]ooW`hq[Z (1367) in=40b,setxattr _Ybhal\\]RPkmpUPPRRb[_[YfQpj[aXsgkfNbeeOZ (946) in=40b,setxattr YeQo_aSrmVSirmpTkkZqfZaaqOtg\\X\\qoRPrjPai (1786) in=40b] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.894746643000001, 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096219809999999999, 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 390 2024-09-17T17:30:27.248 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.249 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332418+0000", 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3379999999999999e-06 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332463+0000", 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4666999999999998e-05 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332482+0000", 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8822e-05 2024-09-17T17:30:27.250 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332493+0000", 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0686000000000001e-05 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332566+0000", 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3250999999999994e-05 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.251 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332571+0000", 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1479999999999997e-06 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.335092+0000", 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0025209799999999999 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.252 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.335166+0000", 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3672999999999997e-05 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.336267+0000", 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011013399999999999 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.338033+0000", 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001766323 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.253 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.341931+0000", 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0038977349999999998 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342038+0000", 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107018 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:395 3.as0 3:5280ab18:::smithi10238019-18:head [append 0~622592] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:27.254 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.342310+0000", 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.884852725000002, 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090640670000000003, 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 395 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.255 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342310+0000", 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342310+0000", 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342312+0000", 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1780000000000002e-06 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342989+0000", 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067656599999999997 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343006+0000", 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7578999999999999e-05 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343015+0000", 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3629999999999994e-06 2024-09-17T17:30:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343065+0000", 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9790000000000003e-05 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343157+0000", 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2149000000000002e-05 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.258 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.344029+0000", 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00087232900000000003 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.348159+0000", 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041304050000000002 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.351310+0000", 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0031507710000000001 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.259 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.351374+0000", 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3936999999999994e-05 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:396 3.as0 3:5280ab18:::smithi10238019-18:head [append 0~147456] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.883973298000001, 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012450536999999999, 2024-09-17T17:30:27.260 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 396 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.261 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343191+0000", 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0839999999999999e-06 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343250+0000", 2024-09-17T17:30:27.262 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9113999999999997e-05 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343262+0000", 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2649999999999999e-05 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.264 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343270+0000", 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0879999999999995e-06 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.345733+0000", 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024625039999999999 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.345807+0000", 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3769000000000005e-05 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.265 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346281+0000", 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047393899999999999 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.350852+0000", 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0045715920000000002 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355563+0000", 2024-09-17T17:30:27.266 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047105319999999999 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355640+0000", 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.7265e-05 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:397 3.as0 3:5280ab18:::smithi10238019-18:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:27.267 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.883873020999999, 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012559297000000001, 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 397 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.268 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3399999999999999e-07 2024-09-17T17:30:27.269 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343293+0000", 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1700000000000001e-06 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343299+0000", 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3199999999999999e-06 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.270 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343302+0000", 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.495e-06 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346935+0000", 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0036331419999999998 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346988+0000", 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2828999999999999e-05 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.271 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347263+0000", 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000274999 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.350964+0000", 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003701101 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355764+0000", 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047997869999999998 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.272 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355849+0000", 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4919999999999993e-05 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:398 3.as0 3:5280ab18:::smithi10238019-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 25.883854933999999, 2024-09-17T17:30:27.273 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013718741, 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 398 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.274 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9700000000000003e-07 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.275 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343310+0000", 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.525e-06 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343314+0000", 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8770000000000003e-06 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343316+0000", 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8080000000000001e-06 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.276 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347523+0000", 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042062560000000002 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347583+0000", 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0201999999999997e-05 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356234+0000", 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086509659999999995 2024-09-17T17:30:27.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356431+0000", 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00019758699999999999 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.357027+0000", 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00059522299999999998 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 51/32 ECSubWriteReply(tid=64, last_complete=51'43, committed=1, applied=1))", 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 24.870388426000002, 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.78010271600000003, 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.279 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356775+0000", 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5000000000000001e-07 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.280 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356781+0000", 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.716e-06 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136139+0000", 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.77935822499999996 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136153+0000", 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.359e-05 2024-09-17T17:30:27.281 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136325+0000", 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017257700000000001 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136877+0000", 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00055165800000000001 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.282 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.8s6 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.337456+0000", 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.889706424, 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0093714709999999993, 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337456+0000", 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.283 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337456+0000", 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337457+0000", 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.92e-07 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337477+0000", 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0305999999999998e-05 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.284 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337511+0000", 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4422999999999999e-05 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337519+0000", 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0369999999999995e-06 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337574+0000", 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5140999999999999e-05 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337585+0000", 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0743e-05 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.346797+0000", 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0092117180000000007 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.346828+0000", 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0911000000000003e-05 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.286 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s2 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.889233936, 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012584702999999999, 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.287 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:30:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337956+0000", 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6574e-05 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338017+0000", 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1050999999999995e-05 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338024+0000", 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2300000000000002e-06 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338194+0000", 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017029299999999999 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338202+0000", 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.892e-06 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.350495+0000", 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012292684 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.350514+0000", 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8678999999999999e-05 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.291 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:641 3.as0 3:5280ab18:::smithi10238019-18:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 7.7034648160000003, 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.16562932699999999, 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 641 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523700+0000", 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1150000000000001e-06 2024-09-17T17:30:27.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523708+0000", 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5599999999999996e-06 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523725+0000", 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7725999999999999e-05 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523733+0000", 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5299999999999999e-06 2024-09-17T17:30:27.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6327000000000004e-05 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523914+0000", 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114837 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524640+0000", 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072579000000000003 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526382+0000", 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0017423060000000001 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689252+0000", 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.162869823 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689327+0000", 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5313000000000002e-05 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:655 3.as0 3:5673cc6d:::smithi10238019-2:head [read 0~1818624,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347150+0000", 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.8800124220000001, 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096743809999999993, 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.297 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 655 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347150+0000", 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.298 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347150+0000", 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347151+0000", 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0900000000000002e-07 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347154+0000", 2024-09-17T17:30:27.299 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0230000000000001e-06 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347161+0000", 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.046e-06 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347166+0000", 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9919999999999998e-06 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.300 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347342+0000", 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017651599999999999 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347346+0000", 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1609999999999999e-06 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351287+0000", 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0039411480000000002 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.301 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351386+0000", 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9679000000000006e-05 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.356825+0000", 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0054383069999999999 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.302 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:661 3.as0 3:5a070e56:::smithi10238019-47:head [copy-from ver 88] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.8799867729999997, 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.026523057999999999, 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 661 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.303 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347177+0000", 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:30:27.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347181+0000", 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5050000000000001e-06 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347187+0000", 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1990000000000003e-06 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.305 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347192+0000", 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2969999999999997e-06 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347359+0000", 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016719099999999999 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347361+0000", 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7039999999999999e-06 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.306 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351847+0000", 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.004485274 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351930+0000", 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2813999999999997e-05 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.367807+0000", 2024-09-17T17:30:27.307 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015877189999999999 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.370285+0000", 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024778510000000001 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.373602+0000", 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0033175460000000002 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.308 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.373699+0000", 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6977000000000003e-05 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:662 3.as0 3:5a070e56:::smithi10238019-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.8799144940000003, 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.022338108999999998, 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.309 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 662 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347249+0000", 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2099999999999997e-07 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347251+0000", 2024-09-17T17:30:27.311 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6589999999999999e-06 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347258+0000", 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4330000000000001e-06 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347263+0000", 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3979999999999998e-06 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.312 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347385+0000", 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000121845 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347390+0000", 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.147e-06 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352716+0000", 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053252270000000001 2024-09-17T17:30:27.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352767+0000", 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1344999999999997e-05 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369400+0000", 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016633254 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.314 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369488+0000", 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7441999999999994e-05 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369586+0000", 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8838000000000005e-05 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:663 3.as0 3:5ee91470:::smithi10238019-6:head [delete] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e69)", 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:27.315 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.8798921140000004, 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016232469999999999, 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 663 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.316 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1300000000000001e-07 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.317 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347273+0000", 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2800000000000002e-06 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347279+0000", 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2170000000000002e-06 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347283+0000", 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4719999999999997e-06 2024-09-17T17:30:27.318 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347416+0000", 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013269700000000001 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347421+0000", 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1019999999999996e-06 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352795+0000", 2024-09-17T17:30:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053736970000000002 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352873+0000", 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8318000000000005e-05 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.353539+0000", 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00066570400000000004 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.356943+0000", 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034039679999999998 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.363452+0000", 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.006509384 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.321 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.363503+0000", 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1218000000000003e-05 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 73/32 ECSubWriteReply(tid=139, last_complete=73'108, committed=1, applied=1))", 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 2.4849449830000001, 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.257005174, 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.322 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.323 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742222+0000", 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4359999999999998e-06 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998674+0000", 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.25645215799999999 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.324 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998683+0000", 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998749+0000", 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5689000000000001e-05 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.999223+0000", 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047410699999999998 2024-09-17T17:30:27.325 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:736 3.as0 3:5ceb0529:::smithi10238019-12:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.843178202, 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.95058956999999999, 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.326 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 736 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.327 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.44e-07 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383988+0000", 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.926e-06 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383994+0000", 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2729999999999999e-06 2024-09-17T17:30:27.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384000+0000", 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.237e-06 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385207+0000", 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001207079 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.329 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385300+0000", 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3450999999999997e-05 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385740+0000", 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000440312 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386613+0000", 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00087278900000000001 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:27.330 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334485+0000", 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94787173599999996 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334574+0000", 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9222999999999996e-05 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:737 3.as0 3:5ceb0529:::smithi10238019-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:27.331 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.8431553030000001, 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.95178945199999998, 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 737 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:27.332 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384008+0000", 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5299999999999999e-07 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.333 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384011+0000", 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.852e-06 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384018+0000", 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3860000000000001e-06 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384023+0000", 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7790000000000003e-06 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386679+0000", 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0026562550000000002 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386739+0000", 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0207999999999997e-05 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334966+0000", 2024-09-17T17:30:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94822698000000005 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.335064+0000", 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7929000000000004e-05 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.335797+0000", 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00073260999999999997 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:27.337 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:27.337 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:27.338 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:27.339 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:27.339 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:27.339 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:27.340 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:27.341 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:27.337+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:27.342 INFO:tasks.rados.rados.0.smithi102.stdout:276: done (1 left) 2024-09-17T17:30:27.342 INFO:tasks.rados.rados.0.smithi102.stdout:280: done (0 left) 2024-09-17T17:30:27.342 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 15 current snap is 22 2024-09-17T17:30:27.342 INFO:tasks.rados.rados.0.smithi102.stdout:281: seq_num 130 ranges {3686400=638976,4325376=671744,4997120=802816,5799936=81920} 2024-09-17T17:30:27.357 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 2] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:30:27.357 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:27.357 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-17T17:30:27.357 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T17:30:27.357 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T17:30:27.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-17T17:30:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:27.358 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:27.363 INFO:tasks.rados.rados.0.smithi102.stdout:281: writing smithi10238019-15 from 3686400 to 4325376 tid 1 2024-09-17T17:30:27.368 INFO:tasks.rados.rados.0.smithi102.stdout:281: writing smithi10238019-15 from 4325376 to 4997120 tid 2 2024-09-17T17:30:27.373 INFO:tasks.rados.rados.0.smithi102.stdout:281: writing smithi10238019-15 from 4997120 to 5799936 tid 3 2024-09-17T17:30:27.373 INFO:tasks.rados.rados.0.smithi102.stdout:281: writing smithi10238019-15 from 5799936 to 5881856 tid 4 2024-09-17T17:30:27.373 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 7 current snap is 22 2024-09-17T17:30:27.373 INFO:tasks.rados.rados.0.smithi102.stdout:282: seq_num 131 ranges {4784128=655360,5439488=425984,5865472=638976,6504448=491520} 2024-09-17T17:30:27.380 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:27.380 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:27.401 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:27.403 INFO:tasks.rados.rados.0.smithi102.stdout:282: writing smithi10238019-7 from 4784128 to 5439488 tid 1 2024-09-17T17:30:27.405 INFO:tasks.rados.rados.0.smithi102.stdout:282: writing smithi10238019-7 from 5439488 to 5865472 tid 2 2024-09-17T17:30:27.409 INFO:tasks.rados.rados.0.smithi102.stdout:282: writing smithi10238019-7 from 5865472 to 6504448 tid 3 2024-09-17T17:30:27.414 INFO:tasks.rados.rados.0.smithi102.stdout:282: writing smithi10238019-7 from 6504448 to 6995968 tid 4 2024-09-17T17:30:27.414 INFO:tasks.rados.rados.0.smithi102.stdout:283: read oid 6 snap -1 2024-09-17T17:30:27.414 INFO:tasks.rados.rados.0.smithi102.stdout:283: expect deleted 2024-09-17T17:30:27.414 INFO:tasks.rados.rados.0.smithi102.stdout:284: snap_create 2024-09-17T17:30:27.414 INFO:tasks.rados.rados.0.smithi102.stdout:281: finishing write tid 1 to smithi10238019-15 2024-09-17T17:30:27.415 INFO:tasks.rados.rados.0.smithi102.stdout:281: finishing write tid 2 to smithi10238019-15 2024-09-17T17:30:27.415 INFO:tasks.rados.rados.0.smithi102.stdout:281: finishing write tid 3 to smithi10238019-15 2024-09-17T17:30:27.415 INFO:tasks.rados.rados.0.smithi102.stdout:281: finishing write tid 4 to smithi10238019-15 2024-09-17T17:30:27.415 INFO:tasks.rados.rados.0.smithi102.stdout:281: finishing write tid 5 to smithi10238019-15 2024-09-17T17:30:27.415 INFO:tasks.rados.rados.0.smithi102.stdout:281: finishing write tid 6 to smithi10238019-15 2024-09-17T17:30:27.415 INFO:tasks.rados.rados.0.smithi102.stdout:281: oid 15 updating version 0 to 59 2024-09-17T17:30:27.416 INFO:tasks.rados.rados.0.smithi102.stdout:281: oid 15 updating version 59 to 61 2024-09-17T17:30:27.416 INFO:tasks.rados.rados.0.smithi102.stdout:281: oid 15 version 61 is already newer than 57 2024-09-17T17:30:27.416 INFO:tasks.rados.rados.0.smithi102.stdout:281: oid 15 version 61 is already newer than 58 2024-09-17T17:30:27.416 INFO:tasks.rados.rados.0.smithi102.stdout:281: oid 15 version 61 is already newer than 60 2024-09-17T17:30:27.416 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 15 v 61 (ObjNum 130 snap 22 seq_num 130) dirty exists 2024-09-17T17:30:27.417 INFO:tasks.rados.rados.0.smithi102.stdout:281: left oid 15 (ObjNum 130 snap 22 seq_num 130) 2024-09-17T17:30:27.417 INFO:tasks.rados.rados.0.smithi102.stdout:282: finishing write tid 1 to smithi10238019-7 2024-09-17T17:30:27.417 INFO:tasks.rados.rados.0.smithi102.stdout:282: finishing write tid 2 to smithi10238019-7 2024-09-17T17:30:27.419 INFO:tasks.rados.rados.0.smithi102.stdout:282: finishing write tid 3 to smithi10238019-7 2024-09-17T17:30:27.419 INFO:tasks.rados.rados.0.smithi102.stdout:282: finishing write tid 4 to smithi10238019-7 2024-09-17T17:30:27.419 INFO:tasks.rados.rados.0.smithi102.stdout:282: finishing write tid 5 to smithi10238019-7 2024-09-17T17:30:27.419 INFO:tasks.rados.rados.0.smithi102.stdout:282: finishing write tid 6 to smithi10238019-7 2024-09-17T17:30:27.419 INFO:tasks.rados.rados.0.smithi102.stdout:282: oid 7 updating version 0 to 64 2024-09-17T17:30:27.419 INFO:tasks.rados.rados.0.smithi102.stdout:282: oid 7 updating version 64 to 66 2024-09-17T17:30:27.420 INFO:tasks.rados.rados.0.smithi102.stdout:282: oid 7 updating version 66 to 67 2024-09-17T17:30:27.420 INFO:tasks.rados.rados.0.smithi102.stdout:282: oid 7 version 67 is already newer than 65 2024-09-17T17:30:27.420 INFO:tasks.rados.rados.0.smithi102.stdout:282: oid 7 updating version 67 to 68 2024-09-17T17:30:27.420 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 68 (ObjNum 131 snap 22 seq_num 131) dirty exists 2024-09-17T17:30:27.421 INFO:tasks.rados.rados.0.smithi102.stdout:282: left oid 7 (ObjNum 131 snap 22 seq_num 131) 2024-09-17T17:30:27.424 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:27.438 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:27.437+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:27.444 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:27.453 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:27.454 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:27.462 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:27.468 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:27.473 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:27.473 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T17:30:27.473 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:27.487 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:27.487 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:27.508 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:27.528 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:27.539 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:27.537+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:27.550 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:27.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:27.567+0000 7f58e6ffd640 0 -- 172.21.15.7:0/3149994383 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f58e82214d0 msgr2=0x7f58e82238c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-17T17:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:27.608 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:27.608 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:27.617 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:27.640 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:27.638+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:27.719 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:27.719 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:27.728 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:27.741 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:27.739+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:27.831 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:27.832 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:27.840 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:27.841 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:27.840+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:27.943 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:27.941+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:27.944 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:27.944 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:27.953 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:28.043 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:28.041+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:28.058 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.058 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.066 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:28.145 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:28.143+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:28.209 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.209 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.218 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:28.246 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:28.245+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:28.320 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.320 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.330 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:281: done (3 left) 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:282: done (2 left) 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:283: done (1 left) 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:284: done (0 left) 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:285: rollback oid 16 current snap is 23 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 16 to 20 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:286: copy_from oid 14 from oid 36 current snap is 23 2024-09-17T17:30:28.342 INFO:tasks.rados.rados.0.smithi102.stdout:287: snap_remove snap 18 2024-09-17T17:30:28.347 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:28.344+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:28.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:30:28.436 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.436 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:28.447 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:28.446+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:28.548 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:28.547+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:28.556 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.556 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:28.649 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:28.647+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:28.672 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.673 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:28.750 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:28.748+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:28.789 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.789 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:28.852 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:28.849+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:28.905 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:28.906 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:28.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:28.952 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:28.951+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:29.022 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.022 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:29.075 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:29.073+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:29.138 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.138 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:29.154 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:29.152+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:29.256 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:29.255+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:29.293 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.293 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:287: done (2 left) 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:288: delete oid 44 current snap is 23 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:285: finishing rollback tid 0 to smithi10238019-16 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:285: finishing rollback tid 1 to smithi10238019-16 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:286: finishing copy_from racing read to smithi10238019-14 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:285: finishing rollback tid 2 to smithi10238019-16 2024-09-17T17:30:29.343 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 72 (ObjNum 95 snap 11 seq_num 95) dirty exists 2024-09-17T17:30:29.344 INFO:tasks.rados.rados.0.smithi102.stdout:286: finishing copy_from to smithi10238019-14 2024-09-17T17:30:29.344 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 14 v 58 (ObjNum 119 snap 19 seq_num 119) dirty exists 2024-09-17T17:30:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:30:29.356 INFO:tasks.rados.rados.0.smithi102.stdout:285: done (2 left) 2024-09-17T17:30:29.356 INFO:tasks.rados.rados.0.smithi102.stdout:286: done (1 left) 2024-09-17T17:30:29.356 INFO:tasks.rados.rados.0.smithi102.stdout:288: done (0 left) 2024-09-17T17:30:29.356 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 11 current snap is 23 2024-09-17T17:30:29.356 INFO:tasks.rados.rados.0.smithi102.stdout:289: seq_num 133 ranges {1867776=688128,2555904=720896,3276800=475136,3751936=360448} 2024-09-17T17:30:29.357 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:29.355+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:29.370 INFO:tasks.rados.rados.0.smithi102.stdout:289: writing smithi10238019-11 from 1867776 to 2555904 tid 1 2024-09-17T17:30:29.374 INFO:tasks.rados.rados.0.smithi102.stdout:289: writing smithi10238019-11 from 2555904 to 3276800 tid 2 2024-09-17T17:30:29.377 INFO:tasks.rados.rados.0.smithi102.stdout:289: writing smithi10238019-11 from 3276800 to 3751936 tid 3 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:289: writing smithi10238019-11 from 3751936 to 4112384 tid 4 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:290: rollback oid 19 current snap is 23 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 19 to 22 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:291: rollback oid 45 current snap is 23 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 45 to 20 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:292: read oid 44 snap -1 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:292: expect deleted 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:293: delete oid 42 current snap is 23 2024-09-17T17:30:29.380 INFO:tasks.rados.rados.0.smithi102.stdout:289: finishing write tid 1 to smithi10238019-11 2024-09-17T17:30:29.382 INFO:tasks.rados.rados.0.smithi102.stdout:289: finishing write tid 2 to smithi10238019-11 2024-09-17T17:30:29.383 INFO:tasks.rados.rados.0.smithi102.stdout:290: finishing rollback tid 0 to smithi10238019-19 2024-09-17T17:30:29.384 INFO:tasks.rados.rados.0.smithi102.stdout:289: finishing write tid 3 to smithi10238019-11 2024-09-17T17:30:29.384 INFO:tasks.rados.rados.0.smithi102.stdout:291: finishing rollback tid 1 to smithi10238019-45 2024-09-17T17:30:29.384 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 45 v 0 (ObjNum 33 snap 0 seq_num 2567249328) dirty dne 2024-09-17T17:30:29.385 INFO:tasks.rados.rados.0.smithi102.stdout:289: finishing write tid 4 to smithi10238019-11 2024-09-17T17:30:29.387 INFO:tasks.rados.rados.0.smithi102.stdout:290: finishing rollback tid 1 to smithi10238019-19 2024-09-17T17:30:29.420 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.420 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:29.457 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:29.456+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:29.467 INFO:tasks.rados.rados.0.smithi102.stdout:289: finishing write tid 5 to smithi10238019-11 2024-09-17T17:30:29.468 INFO:tasks.rados.rados.0.smithi102.stdout:289: finishing write tid 6 to smithi10238019-11 2024-09-17T17:30:29.468 INFO:tasks.rados.rados.0.smithi102.stdout:289: oid 11 updating version 0 to 124 2024-09-17T17:30:29.468 INFO:tasks.rados.rados.0.smithi102.stdout:289: oid 11 updating version 124 to 128 2024-09-17T17:30:29.468 INFO:tasks.rados.rados.0.smithi102.stdout:289: oid 11 version 128 is already newer than 126 2024-09-17T17:30:29.468 INFO:tasks.rados.rados.0.smithi102.stdout:289: oid 11 version 128 is already newer than 127 2024-09-17T17:30:29.469 INFO:tasks.rados.rados.0.smithi102.stdout:289: oid 11 version 128 is already newer than 125 2024-09-17T17:30:29.469 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 11 v 128 (ObjNum 133 snap 23 seq_num 133) dirty exists 2024-09-17T17:30:29.469 INFO:tasks.rados.rados.0.smithi102.stdout:289: left oid 11 (ObjNum 133 snap 23 seq_num 133) 2024-09-17T17:30:29.537 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.537 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:29.589 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:29.587+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:29.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:29.637 INFO:tasks.rados.rados.0.smithi102.stdout:289: done (4 left) 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:291: done (3 left) 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:292: done (2 left) 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:293: done (1 left) 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:294: copy_from oid 38 from oid 1 current snap is 23 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:295: rollback oid 15 current snap is 23 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 15 to 20 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:296: rollback oid 24 current snap is 23 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 24 to 21 2024-09-17T17:30:29.638 INFO:tasks.rados.rados.0.smithi102.stdout:297: snap_create 2024-09-17T17:30:29.639 INFO:tasks.rados.rados.0.smithi102.stdout:296: finishing rollback tid 1 to smithi10238019-24 2024-09-17T17:30:29.639 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 24 v 0 (ObjNum -1728099072 snap 21879 seq_num 304) dirty dne 2024-09-17T17:30:29.642 INFO:tasks.rados.rados.0.smithi102.stdout:295: finishing rollback tid 0 to smithi10238019-15 2024-09-17T17:30:29.643 INFO:tasks.rados.rados.0.smithi102.stdout:294: finishing copy_from racing read to smithi10238019-38 2024-09-17T17:30:29.645 INFO:tasks.rados.rados.0.smithi102.stdout:295: finishing rollback tid 1 to smithi10238019-15 2024-09-17T17:30:29.645 INFO:tasks.rados.rados.0.smithi102.stdout:294: finishing copy_from to smithi10238019-38 2024-09-17T17:30:29.645 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 38 v 5 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-17T17:30:29.656 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.656 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.658 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:29.657+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:29.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:29.747 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:29.760 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:29.758+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:29.770 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:29.770 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:29.773 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.773 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:29.792 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:29.814 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:29.835 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:29.835 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:29.861 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:29.862 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:29.860+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:29.883 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:29.883 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:29.891 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:29.891 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:29.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:29.904 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:29.905 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:29.925 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:29.925 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:29.947 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:29.947 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:29.962 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:29.960+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:29.969 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:29.969 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:29.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:30.011 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:30.015 INFO:tasks.rados.rados.0.smithi102.stdout:295: finishing rollback tid 2 to smithi10238019-15 2024-09-17T17:30:30.015 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 15 v 67 (ObjNum 122 snap 19 seq_num 122) dirty exists 2024-09-17T17:30:30.020 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.020 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:30.020 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:30.020 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:30.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:30.032 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:30.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:30.031+0000 7efea11a3640 0 -- 172.21.15.7:0/2684366570 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7efe9c21f610 msgr2=0x7efe9c221a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:30.063 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:30.062+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:30.138 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.138 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:30.138 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:30.138 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:30.138 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:30.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:30.190 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:30.188+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 41.775621915999999, 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:30:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:30:30.260 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 41.257355678000003, 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.261 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:30:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.565638155000002, 2024-09-17T17:30:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:30:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:203 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~557056] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.548437243999999, 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0069058180000000002, 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 203 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703971+0000", 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4849999999999999e-06 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704582+0000", 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00061129700000000001 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704592+0000", 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0094e-05 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704598+0000", 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6019999999999999e-06 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704655+0000", 2024-09-17T17:30:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7107000000000001e-05 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704712+0000", 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7435000000000002e-05 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.705382+0000", 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067020899999999995 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706343+0000", 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00096062299999999997 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710818+0000", 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044745519999999997 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710874+0000", 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6413999999999999e-05 2024-09-17T17:30:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.546194872999997, 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:30:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.545807969999998, 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:30:30.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:30:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.545774602999998, 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:30:30.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.541325131999997, 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:367 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~589824] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.913714317, 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012183095999999999, 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 367 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.297 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338693+0000", 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.795e-06 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341609+0000", 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029156210000000002 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341623+0000", 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4583000000000001e-05 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341630+0000", 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2230000000000001e-06 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341691+0000", 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0878000000000002e-05 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341771+0000", 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9308999999999996e-05 2024-09-17T17:30:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343069+0000", 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012979420000000001 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348577+0000", 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0055080229999999999 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350781+0000", 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002204742 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350874+0000", 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2979999999999994e-05 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.302 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:368 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~573440] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.910074300000002, 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0088382370000000005, 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 368 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342332+0000", 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.08e-06 2024-09-17T17:30:30.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342959+0000", 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062686600000000001 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343062+0000", 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010252 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343083+0000", 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1135e-05 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344260+0000", 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011769969999999999 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344389+0000", 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012922399999999999 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345712+0000", 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001323182 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348773+0000", 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030611649999999998 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351091+0000", 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023171590000000001 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351169+0000", 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8909e-05 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:369 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~196608] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:30.308 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.908972591000001, 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0079413130000000002, 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 369 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343434+0000", 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4699999999999999e-06 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343667+0000", 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023281800000000001 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343680+0000", 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2524000000000001e-05 2024-09-17T17:30:30.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343686+0000", 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.19e-06 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346596+0000", 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002909867 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346803+0000", 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00020759199999999999 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348042+0000", 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001238636 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350047+0000", 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002004774 2024-09-17T17:30:30.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351299+0000", 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001252364 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351374+0000", 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5078000000000002e-05 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:370 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.908704847999999, 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0082819799999999996, 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 370 2024-09-17T17:30:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3799999999999997e-07 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343710+0000", 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2570000000000007e-06 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343715+0000", 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6829999999999996e-06 2024-09-17T17:30:30.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343721+0000", 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349122+0000", 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005400841 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349229+0000", 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107076 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349735+0000", 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050621599999999998 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351781+0000", 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020453279999999999 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351908+0000", 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012790499999999999 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351983+0000", 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4154999999999997e-05 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.320 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:371 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.908677805, 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010270708, 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 371 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4799999999999999e-07 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343730+0000", 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4389999999999999e-06 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343735+0000", 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1919999999999998e-06 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343738+0000", 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1360000000000001e-06 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352473+0000", 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0087356429999999995 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352570+0000", 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6261999999999994e-05 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353998+0000", 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001428688 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s1 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.337395+0000", 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.915010093999999, 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.006904136, 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337395+0000", 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337395+0000", 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337396+0000", 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1e-07 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337420+0000", 2024-09-17T17:30:30.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3828e-05 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337512+0000", 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2149000000000002e-05 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337520+0000", 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.653e-06 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337598+0000", 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7371000000000002e-05 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.337609+0000", 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1567e-05 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344184+0000", 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0065750189999999997 2024-09-17T17:30:30.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.344300+0000", 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000115239 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.fs6 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:30.330 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.872027562, 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75327243099999996, 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:30.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380379+0000", 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8100000000000002e-07 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380396+0000", 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7717e-05 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132975+0000", 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75257808000000004 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132981+0000", 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2249999999999997e-06 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133046+0000", 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4944999999999996e-05 2024-09-17T17:30:30.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133053+0000", 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133623+0000", 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057078100000000002 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133650+0000", 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6996e-05 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:495 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 21.594341433, 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.022462405000000001, 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 495 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658065+0000", 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2600000000000001e-07 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658072+0000", 2024-09-17T17:30:30.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8430000000000004e-06 2024-09-17T17:30:30.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658082+0000", 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.927e-06 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658089+0000", 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8909999999999998e-06 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660406+0000", 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023178890000000001 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660504+0000", 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7341000000000002e-05 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661132+0000", 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062810899999999996 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661861+0000", 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072939299999999995 2024-09-17T17:30:30.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680403+0000", 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018541588000000001 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680526+0000", 2024-09-17T17:30:30.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012359799999999999 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/35 ECSubWriteReply(tid=37, last_complete=57'27, committed=1, applied=1))", 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 21.590574607000001, 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019253007999999999, 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661832+0000", 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661839+0000", 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7999999999999999e-06 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680112+0000", 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018272335000000001 2024-09-17T17:30:30.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680126+0000", 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3867e-05 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680275+0000", 2024-09-17T17:30:30.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000149741 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.681084+0000", 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00080847500000000001 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:30.348 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.813541494999999, 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0071050749999999998, 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3699999999999999e-07 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438897+0000", 2024-09-17T17:30:30.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3120999999999997e-05 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438953+0000", 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5955000000000002e-05 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438970+0000", 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6263e-05 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441711+0000", 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027414589999999999 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441722+0000", 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0797e-05 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.445920+0000", 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0041985929999999996 2024-09-17T17:30:30.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.445969+0000", 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8649999999999997e-05 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=196, reqid=client.4566.0:639, at_version=68'102, trim_to=0'0, roll_forward_to=68'97))", 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.726669169999999, 2024-09-17T17:30:30.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.667795892, 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:30.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525737+0000", 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525756+0000", 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9026999999999999e-05 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192770+0000", 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.66701338300000002 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192793+0000", 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3203000000000001e-05 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192903+0000", 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109973 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192912+0000", 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9139999999999997e-06 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193502+0000", 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058984700000000001 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193532+0000", 2024-09-17T17:30:30.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0651999999999997e-05 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=197, reqid=client.4566.0:639, at_version=0'0, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.058496672, 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15173107799999999, 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5799999999999999e-07 2024-09-17T17:30:30.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193938+0000", 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8988000000000001e-05 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344048+0000", 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15010903 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344059+0000", 2024-09-17T17:30:30.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1148000000000001e-05 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344595+0000", 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00053675200000000004 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344608+0000", 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2211e-05 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345601+0000", 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00099356800000000001 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345640+0000", 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8723000000000002e-05 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.362 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:30.364 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:30.364 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:30.365 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:30.366 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:294: done (4 left) 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:295: done (3 left) 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:296: done (2 left) 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:297: done (1 left) 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 27 current snap is 24 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:298: seq_num 135 ranges {1785856=720896,2506752=622592,3129344=688128,3817472=245760} 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:298: writing smithi10238019-27 from 1785856 to 2506752 tid 1 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:298: writing smithi10238019-27 from 2506752 to 3129344 tid 2 2024-09-17T17:30:30.367 INFO:tasks.rados.rados.0.smithi102.stdout:298: writing smithi10238019-27 from 3129344 to 3817472 tid 3 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:298: writing smithi10238019-27 from 3817472 to 4063232 tid 4 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:299: read oid 41 snap -1 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:299: expect (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:300: read oid 24 snap -1 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:300: expect deleted 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:301: copy_from oid 38 from oid 23 current snap is 24 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:290: finishing rollback tid 2 to smithi10238019-19 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 19 v 64 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:290: done (4 left) 2024-09-17T17:30:30.368 INFO:tasks.rados.rados.0.smithi102.stdout:302: snap_remove snap 23 2024-09-17T17:30:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:30.369 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:30.362+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:30.465 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:30.463+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:30.514 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.514 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:30.514 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:30.514 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:30.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:30.565 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:30.564+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:30.630 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.630 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:30.630 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:30.630 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:30.630 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:30.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:30.683 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:30.681+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:30.749 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 41.753366620000001, 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:30:30.751 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:30:30.753 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:30.754 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 40.024430314999996, 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002528565, 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720703+0000", 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5300000000000004e-07 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721318+0000", 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000615779 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721351+0000", 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2486e-05 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721360+0000", 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8729999999999996e-06 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721434+0000", 2024-09-17T17:30:30.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4011999999999995e-05 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721445+0000", 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0903999999999999e-05 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723198+0000", 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001753641 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723231+0000", 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2217e-05 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 37.019293580000003, 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.760 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:30:30.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.051265708000003, 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.764 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:30:30.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.051023598, 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:30:30.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:30.773 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.355489459000001, 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027457129999999999, 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2200000000000004e-07 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390408+0000", 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00076470399999999995 2024-09-17T17:30:30.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390444+0000", 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5933000000000003e-05 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390453+0000", 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5120000000000008e-06 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390525+0000", 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2849000000000007e-05 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390537+0000", 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1999e-05 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392350+0000", 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018122660000000001 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392389+0000", 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8828000000000002e-05 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/32 ECSubWrite(tid=110, reqid=client.4566.0:436, at_version=52'44, trim_to=0'0, roll_forward_to=52'42))", 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.343678646000001, 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002489495, 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3799999999999998e-07 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402030+0000", 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000576261 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402060+0000", 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9512000000000001e-05 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402068+0000", 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7970000000000001e-06 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402133+0000", 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5083e-05 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402155+0000", 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1906999999999999e-05 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.403904+0000", 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017489510000000001 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.403943+0000", 2024-09-17T17:30:30.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9545999999999997e-05 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.406874151, 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024521180000000001, 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.783 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.22e-07 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338285+0000", 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.686e-05 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338389+0000", 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000103298 2024-09-17T17:30:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338398+0000", 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5170000000000005e-06 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338479+0000", 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1052999999999998e-05 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338489+0000", 2024-09-17T17:30:30.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6889999999999999e-06 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.340666+0000", 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021771529999999998 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.340710+0000", 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4226e-05 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.360531780999999, 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029370899999999998, 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.788 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384601+0000", 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8e-07 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385325+0000", 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072435200000000005 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385406+0000", 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0667999999999994e-05 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385415+0000", 2024-09-17T17:30:30.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385509+0000", 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4137999999999999e-05 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385519+0000", 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0208e-05 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387496+0000", 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001976387 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387538+0000", 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1996999999999998e-05 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.792 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.359803619000001, 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029662, 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385362+0000", 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2403999999999997e-05 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385471+0000", 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109485 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385479+0000", 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6750000000000002e-06 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.795 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386169+0000", 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068989099999999999 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386179+0000", 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0278000000000001e-05 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388245+0000", 2024-09-17T17:30:30.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020665150000000001 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388295+0000", 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9369000000000001e-05 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 59/32 ECSubWrite(tid=146, reqid=client.4566.0:504, at_version=59'61, trim_to=0'0, roll_forward_to=59'59))", 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:10.356814+0000", 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 20.388318031000001, 2024-09-17T17:30:30.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025928259999999999, 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.356814+0000", 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.356814+0000", 2024-09-17T17:30:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.356815+0000", 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9899999999999999e-07 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357476+0000", 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00066073 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357512+0000", 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5945999999999998e-05 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357522+0000", 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7790000000000007e-06 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357577+0000", 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5819999999999997e-05 2024-09-17T17:30:30.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357588+0000", 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0118999999999999e-05 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.359376+0000", 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001788127 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.359407+0000", 2024-09-17T17:30:30.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1405999999999997e-05 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 65/32 ECSubWrite(tid=171, reqid=client.4566.0:584, at_version=65'81, trim_to=0'0, roll_forward_to=62'79))", 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 14.378202236, 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023842849999999999, 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4799999999999999e-07 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366961+0000", 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0667999999999998e-05 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367046+0000", 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5097000000000002e-05 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367056+0000", 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.533e-06 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367138+0000", 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1819999999999999e-05 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367149+0000", 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1131000000000001e-05 2024-09-17T17:30:30.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.369264+0000", 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021152010000000001 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.369314+0000", 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0386999999999997e-05 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.306056519, 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029678690000000001, 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:30.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1899999999999998e-07 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439101+0000", 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4655e-05 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439183+0000", 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2472000000000006e-05 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439193+0000", 2024-09-17T17:30:30.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0063e-05 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439257+0000", 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3775000000000005e-05 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439270+0000", 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2588e-05 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441995+0000", 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002725505 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442044+0000", 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8392e-05 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/67 ECSubWrite(tid=150, reqid=osd.6.0:150, at_version=68'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:30.811 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.29645272, 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023107700000000002, 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8999999999999995e-07 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448709+0000", 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8249999999999999e-05 2024-09-17T17:30:30.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448753+0000", 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4716999999999999e-05 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448765+0000", 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1959e-05 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448863+0000", 2024-09-17T17:30:30.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7893000000000006e-05 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448881+0000", 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7997000000000001e-05 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.450945+0000", 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020640820000000001 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.450990+0000", 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4981999999999997e-05 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=188, reqid=client.4566.0:623, at_version=68'94, trim_to=0'0, roll_forward_to=68'92))", 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.492843+0000", 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.252288812, 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002728776, 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492843+0000", 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492843+0000", 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492846+0000", 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.323e-06 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493606+0000", 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00076045400000000003 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493649+0000", 2024-09-17T17:30:30.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2842000000000001e-05 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493659+0000", 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0179e-05 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493730+0000", 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1118000000000006e-05 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493744+0000", 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3348e-05 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495531+0000", 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001786986 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495572+0000", 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1526000000000002e-05 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 69/32 ECSubWrite(tid=56, reqid=client.4566.0:656, at_version=69'42, trim_to=0'0, roll_forward_to=65'40))", 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.393271325000001, 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026116289999999999, 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.48e-07 2024-09-17T17:30:30.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351878+0000", 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7079e-05 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351932+0000", 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4051000000000001e-05 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351944+0000", 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1654999999999999e-05 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.352108+0000", 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016444800000000001 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.352129+0000", 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0545000000000001e-05 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.354429+0000", 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023003580000000002 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.354473+0000", 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3344999999999999e-05 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 70/67 ECSubWrite(tid=155, reqid=osd.6.0:155, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.3999638480000005, 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00308939, 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345169+0000", 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345192+0000", 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3241e-05 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345239+0000", 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6607999999999998e-05 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345247+0000", 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5339999999999999e-06 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345711+0000", 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00046389400000000001 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345720+0000", 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0710000000000001e-06 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348210+0000", 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024892099999999999 2024-09-17T17:30:30.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348258+0000", 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8208000000000003e-05 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=199, reqid=osd.11.0:199, at_version=70'104, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.3996698409999997, 2024-09-17T17:30:30.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002830782, 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:30.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345463+0000", 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345486+0000", 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3453999999999999e-05 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345559+0000", 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2760000000000001e-05 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345586+0000", 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7670000000000001e-05 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345650+0000", 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3139999999999995e-05 2024-09-17T17:30:30.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345661+0000", 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1342000000000001e-05 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348252+0000", 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025906829999999999 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348293+0000", 2024-09-17T17:30:30.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.155e-05 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=200, reqid=osd.11.0:200, at_version=70'106, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.3990884070000007, 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027884030000000001, 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.835 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346045+0000", 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346081+0000", 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5942999999999998e-05 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346125+0000", 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3949999999999998e-05 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346134+0000", 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5019999999999998e-06 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346909+0000", 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077504099999999997 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346921+0000", 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1598e-05 2024-09-17T17:30:30.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348789+0000", 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001868268 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348832+0000", 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3059000000000002e-05 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 76/32 ECSubWrite(tid=67, reqid=client.4566.0:740, at_version=76'57, trim_to=0'0, roll_forward_to=73'55))", 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:27.364684+0000", 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 3.3804477689999999, 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023389719999999999, 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.364684+0000", 2024-09-17T17:30:30.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.364684+0000", 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.364685+0000", 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7300000000000002e-07 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:30.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365172+0000", 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048696299999999998 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365208+0000", 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6180999999999998e-05 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365217+0000", 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.799e-06 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365286+0000", 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9776000000000004e-05 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365297+0000", 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1018e-05 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.366991+0000", 2024-09-17T17:30:30.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0016936169999999999 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.367023+0000", 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2345000000000003e-05 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:30.844 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:30.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:30.846 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:30.844+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:30.947 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:30.945+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:30.993 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:30.994 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:30.994 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:30.994 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:31.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:31.047 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:31.045+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:31.110 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:31.110 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:31.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:31.149 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:31.146+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:31.228 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:31.228 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:31.228 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:31.228 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:31.228 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 43/32 ECSubWrite(tid=18, reqid=osd.7.0:18, at_version=43'14, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.532824300000001, 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025590249999999999, 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692918+0000", 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1799999999999999e-07 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.692948+0000", 2024-09-17T17:30:31.230 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9323999999999999e-05 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693000+0000", 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1894999999999997e-05 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693011+0000", 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1481000000000001e-05 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.231 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693101+0000", 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0237999999999999e-05 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693114+0000", 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2325e-05 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695440+0000", 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023263030000000001 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.695477+0000", 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7240999999999998e-05 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.531682148000002, 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026508170000000002, 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.234 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694061+0000", 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5099999999999998e-07 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694087+0000", 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5571999999999998e-05 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.235 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694133+0000", 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5583e-05 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694142+0000", 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6500000000000008e-06 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694227+0000", 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5159000000000002e-05 2024-09-17T17:30:31.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694246+0000", 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8723e-05 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696667+0000", 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024210709999999999 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696711+0000", 2024-09-17T17:30:31.237 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4107999999999998e-05 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.836104370000001, 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026689740000000002, 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.238 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389639+0000", 2024-09-17T17:30:31.239 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.06e-07 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390292+0000", 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00065341900000000003 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390322+0000", 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.921e-05 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.240 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390328+0000", 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5599999999999999e-06 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390385+0000", 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6484999999999999e-05 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390396+0000", 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1409e-05 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.241 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392271+0000", 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001875383 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392307+0000", 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5802e-05 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.242 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.886518685999999, 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032487919999999999, 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339225+0000", 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6799999999999999e-07 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339248+0000", 2024-09-17T17:30:31.244 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3654000000000001e-05 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339288+0000", 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9734999999999998e-05 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339297+0000", 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.245 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339672+0000", 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037528099999999999 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339682+0000", 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0168e-05 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342436+0000", 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027540889999999999 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342473+0000", 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6331999999999997e-05 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.886340586999999, 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030245340000000002, 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.247 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0100000000000001e-07 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339459+0000", 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6342999999999999e-05 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339692+0000", 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023318199999999999 2024-09-17T17:30:31.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339701+0000", 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7469999999999994e-06 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339766+0000", 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5277999999999994e-05 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.250 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339776+0000", 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8439999999999993e-06 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342401+0000", 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026253349999999999 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342426+0000", 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5503999999999998e-05 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.251 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=50, reqid=osd.7.0:49, at_version=0'0, trim_to=0'0, roll_forward_to=56'40))", 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339764+0000", 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.885978222999999, 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026235939999999999, 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.252 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339764+0000", 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339764+0000", 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339765+0000", 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.68e-07 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339774+0000", 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3239999999999992e-06 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339828+0000", 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3384999999999998e-05 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339836+0000", 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0530000000000007e-06 2024-09-17T17:30:31.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339918+0000", 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1786999999999995e-05 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339953+0000", 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5077000000000003e-05 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342341+0000", 2024-09-17T17:30:31.255 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023879510000000001 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342388+0000", 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7348999999999997e-05 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=88, reqid=client.4566.0:477, at_version=56'35, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353197+0000", 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.872545582000001, 2024-09-17T17:30:31.256 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028115079999999999, 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353197+0000", 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353197+0000", 2024-09-17T17:30:31.257 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353198+0000", 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1999999999999998e-07 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353639+0000", 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044126499999999998 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.258 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353970+0000", 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00033084299999999999 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353980+0000", 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7880000000000001e-06 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354058+0000", 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8360000000000002e-05 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.259 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354069+0000", 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0899e-05 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355985+0000", 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019164970000000001 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356008+0000", 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3036000000000001e-05 2024-09-17T17:30:31.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=90, reqid=client.4566.0:479, at_version=56'37, trim_to=0'0, roll_forward_to=56'32))", 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.871923731999999, 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026104409999999998, 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.261 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3000000000000001e-07 2024-09-17T17:30:31.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353893+0000", 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4086999999999997e-05 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354045+0000", 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00015175099999999999 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354051+0000", 2024-09-17T17:30:31.263 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8980000000000001e-06 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355384+0000", 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013329189999999999 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355390+0000", 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0240000000000003e-06 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.264 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356391+0000", 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010013159999999999 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356429+0000", 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7916000000000001e-05 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.265 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=91, reqid=client.4566.0:480, at_version=56'38, trim_to=0'0, roll_forward_to=56'32))", 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353897+0000", 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.871845346000001, 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027461170000000002, 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353897+0000", 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.266 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353897+0000", 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353898+0000", 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.63e-07 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353909+0000", 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1810999999999999e-05 2024-09-17T17:30:31.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354065+0000", 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00015532 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354072+0000", 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6290000000000001e-06 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355801+0000", 2024-09-17T17:30:31.268 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017290529999999999 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355811+0000", 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8679999999999994e-06 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356611+0000", 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00079965099999999996 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.269 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356643+0000", 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2322e-05 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=92, reqid=osd.12.0:92, at_version=56'40, trim_to=0'0, roll_forward_to=56'34))", 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.871831084, 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003074832, 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.270 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.271 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353912+0000", 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353937+0000", 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5015e-05 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354085+0000", 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014835900000000001 2024-09-17T17:30:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354091+0000", 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8220000000000003e-06 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356082+0000", 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019913650000000001 2024-09-17T17:30:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356088+0000", 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1190000000000002e-06 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356953+0000", 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00086492399999999999 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.274 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356986+0000", 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2994999999999998e-05 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.853289585999999, 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030867720000000002, 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.275 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2800000000000001e-07 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372483+0000", 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9967999999999998e-05 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372565+0000", 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2091999999999999e-05 2024-09-17T17:30:31.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372574+0000", 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8869999999999998e-06 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372659+0000", 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5358999999999994e-05 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372674+0000", 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4827000000000001e-05 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375499+0000", 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028244870000000001 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375540+0000", 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1023999999999999e-05 2024-09-17T17:30:31.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.852038913000001, 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033844539999999998, 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373730+0000", 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5689000000000001e-05 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373771+0000", 2024-09-17T17:30:31.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1208000000000003e-05 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374141+0000", 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037011200000000002 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374273+0000", 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013144900000000001 2024-09-17T17:30:31.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374283+0000", 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0147e-05 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377050+0000", 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002767346 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377088+0000", 2024-09-17T17:30:31.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7713000000000003e-05 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "pg_update_log_missing(3.cs6 epoch 59/32 rep_tid 142 entries 59'57 (0'0) error 3:3116c96c:::smithi10238019-11:head by client.4566.0:501 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 57'54)", 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 20.887241699, 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027076629999999999, 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.285 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.03e-07 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338508+0000", 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1520000000000003e-06 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.339188+0000", 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068008399999999996 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.339207+0000", 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8369999999999999e-05 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.339358+0000", 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000151144 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.341209+0000", 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00185071 2024-09-17T17:30:31.288 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 64/35 ECSubWrite(tid=108, reqid=client.4566.0:565, at_version=64'40, trim_to=0'0, roll_forward_to=60'38))", 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 15.888148592, 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027638189999999998, 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.12e-07 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337844+0000", 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00024958599999999999 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337881+0000", 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7255999999999997e-05 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337892+0000", 2024-09-17T17:30:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1274000000000001e-05 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337984+0000", 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1729000000000002e-05 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.338000+0000", 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6157000000000001e-05 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.340311+0000", 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002311006 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.340358+0000", 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6699000000000001e-05 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.785142667000001, 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003708654, 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5499999999999999e-07 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440633+0000", 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2728999999999998e-05 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440769+0000", 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013610200000000001 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440779+0000", 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9410000000000002e-06 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442220+0000", 2024-09-17T17:30:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0014405760000000001 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442231+0000", 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1847e-05 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444268+0000", 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020363299999999998 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444309+0000", 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0774e-05 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 68/67 ECSubWrite(tid=122, reqid=client.4566.0:616, at_version=68'46, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.781669603999999, 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030735390000000001, 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444074+0000", 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1200000000000002e-07 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444177+0000", 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010335700000000001 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444221+0000", 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4370999999999999e-05 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444232+0000", 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0178e-05 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444294+0000", 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2204000000000003e-05 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444303+0000", 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9630000000000004e-06 2024-09-17T17:30:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447110+0000", 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002806809 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447146+0000", 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6944999999999999e-05 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 68/67 ECSubWrite(tid=124, reqid=client.4566.0:618, at_version=68'48, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.463859+0000", 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.761883955, 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025525119999999998, 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.463859+0000", 2024-09-17T17:30:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.463859+0000", 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.463859+0000", 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9699999999999995e-07 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.464402+0000", 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00054288100000000005 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.464448+0000", 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5328999999999999e-05 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.464459+0000", 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1112e-05 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.464527+0000", 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8738999999999994e-05 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.464535+0000", 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5270000000000001e-06 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.466369+0000", 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018335789999999999 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.466411+0000", 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2548e-05 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/32 ECSubWrite(tid=188, reqid=client.4566.0:623, at_version=68'94, trim_to=0'0, roll_forward_to=68'92))", 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.492816+0000", 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.732926818999999, 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027185249999999998, 2024-09-17T17:30:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492816+0000", 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492816+0000", 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492827+0000", 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.129e-05 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493571+0000", 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000743919 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493603+0000", 2024-09-17T17:30:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1671000000000002e-05 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493609+0000", 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.246e-06 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493676+0000", 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6816999999999994e-05 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493686+0000", 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0661e-05 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495499+0000", 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001812676 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495534+0000", 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5244999999999999e-05 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=157, to_read={3:00c2d3b1:::smithi10238019-29:head=0,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 6.4104602350000004, 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30626450500000002, 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815283+0000", 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9299999999999998e-07 2024-09-17T17:30:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815290+0000", 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8709999999999999e-06 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121092+0000", 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30580278100000002 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121102+0000", 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5170000000000005e-06 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121186+0000", 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4229999999999998e-05 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121547+0000", 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00036061300000000003 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=171, to_read={3:00c2d3b1:::smithi10238019-29:head=262144,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 6.101622742, 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.21319038800000001, 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.61e-07 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124129+0000", 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5269999999999998e-06 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336658+0000", 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.212529417 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336671+0000", 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2658e-05 2024-09-17T17:30:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336735+0000", 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4203999999999997e-05 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.337310+0000", 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057492100000000005 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.319 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:31.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:31.322 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:31.318+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:302: done (4 left) 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:303: read oid 40 snap -1 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:303: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:304: snap_create 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:298: finishing write tid 1 to smithi10238019-27 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:298: finishing write tid 2 to smithi10238019-27 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:298: finishing write tid 3 to smithi10238019-27 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:298: finishing write tid 4 to smithi10238019-27 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:299: expect (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:30:31.340 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:31.363 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:31.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:30:31.384 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: finishing write tid 5 to smithi10238019-27 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:301: finishing copy_from racing read to smithi10238019-38 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: finishing write tid 6 to smithi10238019-27 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: oid 27 updating version 0 to 56 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: oid 27 updating version 56 to 58 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: oid 27 updating version 58 to 59 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: oid 27 version 59 is already newer than 57 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: oid 27 updating version 59 to 60 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 27 v 60 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:298: left oid 27 (ObjNum 135 snap 24 seq_num 135) 2024-09-17T17:30:31.406 INFO:tasks.rados.rados.0.smithi102.stdout:301: finishing copy_from to smithi10238019-38 2024-09-17T17:30:31.407 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 38 v 25 (ObjNum 70 snap 4 seq_num 70) dirty exists 2024-09-17T17:30:31.422 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:31.420+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:31.434 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:31.435 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:31.435 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:31.435 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:31.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:31.523 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:31.520+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:31.551 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:31.551 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:31.551 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:31.552 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:31.552 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:31.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:31.624 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:31.621+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:31.669 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:182 3.7s0 3:ea2a6e9a:::smithi10238019-22:head [copy-from ver 8] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 40.988751463, 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.030026536, 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.670 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 182 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.671 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677857+0000", 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2839999999999999e-06 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677863+0000", 2024-09-17T17:30:31.672 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8850000000000004e-06 2024-09-17T17:30:31.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.673 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.673 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677875+0000", 2024-09-17T17:30:31.673 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2490999999999999e-05 2024-09-17T17:30:31.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.674 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.674 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677882+0000", 2024-09-17T17:30:31.674 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.652e-06 2024-09-17T17:30:31.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.675 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.675 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677943+0000", 2024-09-17T17:30:31.675 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1327000000000004e-05 2024-09-17T17:30:31.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678017+0000", 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3440000000000002e-05 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703307+0000", 2024-09-17T17:30:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.025289977000000002 2024-09-17T17:30:31.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.677 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.677 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704581+0000", 2024-09-17T17:30:31.677 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012741790000000001 2024-09-17T17:30:31.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.678 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.678 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707797+0000", 2024-09-17T17:30:31.678 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003216642 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707882+0000", 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4659000000000004e-05 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:183 3.7s0 3:ea2a6e9a:::smithi10238019-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 40.988712751000001, 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026638608000000001, 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 183 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:31.681 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2099999999999998e-07 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677896+0000", 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.804e-06 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677902+0000", 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5149999999999997e-06 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677905+0000", 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5209999999999998e-06 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678152+0000", 2024-09-17T17:30:31.683 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000246536 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678180+0000", 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8605000000000001e-05 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.684 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704308+0000", 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026127128999999999 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704406+0000", 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8079999999999996e-05 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704533+0000", 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012709700000000001 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.685 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:187 3.6s0 3:672dafef:::smithi10238019-1:head [append 0~507904] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.681400+0000", 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 40.985206448, 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01108514, 2024-09-17T17:30:31.686 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.687 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.687 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.687 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 187 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681400+0000", 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681400+0000", 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681402+0000", 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3379999999999999e-06 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681984+0000", 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058243900000000003 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.689 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.681999+0000", 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4763000000000001e-05 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.682011+0000", 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2238999999999999e-05 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.682074+0000", 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2508999999999994e-05 2024-09-17T17:30:31.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.682177+0000", 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000103331 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.683737+0000", 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001560131 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.687180+0000", 2024-09-17T17:30:31.691 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003442793 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.692401+0000", 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0052209300000000004 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.692486+0000", 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4666999999999994e-05 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:190 3.6s0 3:672dafef:::smithi10238019-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.684285+0000", 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 40.982321730999999, 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010327061, 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 190 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.694 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684285+0000", 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684285+0000", 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684286+0000", 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1200000000000002e-07 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684289+0000", 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1970000000000001e-06 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684307+0000", 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8182e-05 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.684315+0000", 2024-09-17T17:30:31.696 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.712e-06 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.687051+0000", 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027358349999999998 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.687129+0000", 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8282999999999996e-05 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.693810+0000", 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0066809240000000004 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.693937+0000", 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012735399999999999 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.694612+0000", 2024-09-17T17:30:31.698 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067486200000000003 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 43/32 ECSubWriteReply(tid=93, last_complete=43'8, committed=1, applied=1))", 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.959478515000001, 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.427260368, 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.699 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.700 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707129+0000", 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9599999999999999e-07 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707133+0000", 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0629999999999999e-06 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.701 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134286+0000", 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.427152792 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134295+0000", 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9500000000000007e-06 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134356+0000", 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1439000000000006e-05 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.702 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134389+0000", 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2428000000000002e-05 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:444 3.6s0 3:682da707:::smithi10238019-8:head [append 0~737280] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.467332+0000", 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 28.199274627000001, 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.011236344000000001, 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 444 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.467332+0000", 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.704 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.467332+0000", 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.467335+0000", 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5660000000000001e-06 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.468053+0000", 2024-09-17T17:30:31.705 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00071794100000000004 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.468072+0000", 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9664999999999999e-05 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.468081+0000", 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6179999999999995e-06 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.706 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.468158+0000", 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7075000000000003e-05 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.468233+0000", 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4838000000000004e-05 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.469056+0000", 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00082293500000000001 2024-09-17T17:30:31.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.470693+0000", 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0016372730000000001 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.478472+0000", 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077786219999999998 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.478569+0000", 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6811000000000005e-05 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:489 3.1s0 3:87ca9cdd:::smithi10238019-41:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.297092816999999, 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01402203, 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 489 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:30:31.710 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369517+0000", 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8480000000000001e-06 2024-09-17T17:30:31.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369527+0000", 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0037000000000001e-05 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369548+0000", 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0653e-05 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.712 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369556+0000", 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5909999999999996e-06 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369630+0000", 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3579999999999997e-05 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "op must wait for map", 2024-09-17T17:30:31.713 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369636+0000", 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6900000000000003e-06 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373208+0000", 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0035712719999999999 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.714 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374240+0000", 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010317569999999999 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375226+0000", 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00098623599999999993 2024-09-17T17:30:31.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.380028+0000", 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0048024870000000002 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.383466+0000", 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0034380299999999999 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.716 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.383536+0000", 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9849000000000002e-05 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:490 3.1s0 3:87ca9cdd:::smithi10238019-41:head [rollback 9] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.297029013, 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016574469000000001, 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 490 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.718 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5999999999999999e-07 2024-09-17T17:30:31.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369581+0000", 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5000000000000002e-06 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369586+0000", 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7570000000000004e-06 2024-09-17T17:30:31.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369589+0000", 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7340000000000002e-06 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369733+0000", 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014407800000000001 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:31.721 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369741+0000", 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8310000000000001e-06 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.376615+0000", 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0068736020000000004 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.376737+0000", 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012183199999999999 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377593+0000", 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00085658399999999997 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.381579+0000", 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039858879999999999 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386071+0000", 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044917280000000004 2024-09-17T17:30:31.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386152+0000", 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1475000000000002e-05 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:491 3.1s0 3:87ca9cdd:::smithi10238019-41:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.29698986, 2024-09-17T17:30:31.724 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016871317, 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 491 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8799999999999998e-07 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369623+0000", 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0730000000000004e-06 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369629+0000", 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0310000000000004e-06 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.727 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369633+0000", 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9899999999999999e-06 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369901+0000", 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00026797300000000002 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369915+0000", 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4143e-05 2024-09-17T17:30:31.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.378961+0000", 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0090466739999999993 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.729 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379126+0000", 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016480800000000001 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379682+0000", 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00055593299999999999 2024-09-17T17:30:31.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.381583+0000", 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001901123 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386406+0000", 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0048232199999999996 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.731 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386488+0000", 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1960999999999997e-05 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.733 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:492 3.bs0 3:d4565dea:::smithi10238019-24:head [delete] snapc c=[9] ondisk+write+known_if_redirected+full_force+supports_pool_eio e57)", 2024-09-17T17:30:31.733 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:30:31.733 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.242889275, 2024-09-17T17:30:31.733 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.23375553299999999, 2024-09-17T17:30:31.733 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.734 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.734 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.734 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 492 2024-09-17T17:30:31.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.735 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.735 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.735 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:30:31.735 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423720+0000", 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4530000000000001e-06 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423731+0000", 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1127e-05 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423762+0000", 2024-09-17T17:30:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1028999999999997e-05 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423777+0000", 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4357e-05 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423949+0000", 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017206400000000001 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.738 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.424109+0000", 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000160703 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.424863+0000", 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00075321999999999997 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.425916+0000", 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001053888 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.657390+0000", 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.231473764 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.740 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.657473+0000", 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2928000000000003e-05 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:515 3.6s0 3:682da707:::smithi10238019-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e59)", 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:10.374753+0000", 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 21.291854011000002, 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010716921000000001, 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 515 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.374753+0000", 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.374753+0000", 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.374753+0000", 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9900000000000002e-07 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.374755+0000", 2024-09-17T17:30:31.743 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.401e-06 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.374759+0000", 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1880000000000004e-06 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.374762+0000", 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3230000000000002e-06 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.744 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.377154+0000", 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023916480000000001 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.377241+0000", 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7114000000000007e-05 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.745 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.384679+0000", 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00743817 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.384772+0000", 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2553000000000006e-05 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.385470+0000", 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00069822500000000002 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.746 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:582 3.6s0 3:672dafef:::smithi10238019-1:head [copy-from ver 5] snapc 11=[11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 15.304652554, 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019590435999999999, 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.747 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 582 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361956+0000", 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4789999999999999e-06 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361969+0000", 2024-09-17T17:30:31.749 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2979999999999999e-05 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361983+0000", 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3917000000000001e-05 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361992+0000", 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8850000000000005e-06 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364412+0000", 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002420082 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364470+0000", 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8588999999999998e-05 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.751 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.379219+0000", 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014748225 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380383+0000", 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011644050000000001 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.381488+0000", 2024-09-17T17:30:31.752 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001105225 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.381545+0000", 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6648999999999999e-05 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:583 3.6s0 3:672dafef:::smithi10238019-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 15.304578654, 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018119641999999998, 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:31.754 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 583 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.755 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362029+0000", 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5799999999999999e-07 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362032+0000", 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2729999999999998e-06 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362040+0000", 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5650000000000004e-06 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362044+0000", 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5549999999999996e-06 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364703+0000", 2024-09-17T17:30:31.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026589909999999999 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364754+0000", 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0785999999999998e-05 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380012+0000", 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015258315 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380076+0000", 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.368e-05 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380148+0000", 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1919e-05 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 68/67 ECSubWriteReply(tid=199, last_complete=68'121, committed=1, applied=1))", 2024-09-17T17:30:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.162616495, 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.188788604, 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.760 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503991+0000", 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7700000000000003e-07 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.761 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503996+0000", 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4039999999999997e-06 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692049+0000", 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.188053784 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692058+0000", 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1480000000000006e-06 2024-09-17T17:30:31.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692105+0000", 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6878000000000001e-05 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692779+0000", 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067451299999999996 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.763 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/67 ECSubWriteReply(tid=202, last_complete=68'119, committed=1, applied=1))", 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 11.966735473, 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.16860020100000001, 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.764 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699872+0000", 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2699999999999998e-07 2024-09-17T17:30:31.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699877+0000", 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7090000000000001e-06 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868220+0000", 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.168342782 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868242+0000", 2024-09-17T17:30:31.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1985999999999999e-05 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868400+0000", 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000158331 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868472+0000", 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1565999999999999e-05 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 70/67 ECSubWriteReply(tid=208, last_complete=70'125, committed=1, applied=1))", 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.302605814, 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.158690363, 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.768 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.769 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.769 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:30:31.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364002+0000", 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1199999999999998e-07 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364005+0000", 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.117e-06 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522493+0000", 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15848867899999999 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522508+0000", 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4419999999999999e-05 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522668+0000", 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016041699999999999 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522691+0000", 2024-09-17T17:30:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3317999999999999e-05 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 70/67 ECSubWriteReply(tid=208, last_complete=70'125, committed=1, applied=1))", 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.298244828, 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.71001705199999998, 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.774 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:30:31.776 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:30:31.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368363+0000", 2024-09-17T17:30:31.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7199999999999997e-07 2024-09-17T17:30:31.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.778 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368367+0000", 2024-09-17T17:30:31.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3680000000000004e-06 2024-09-17T17:30:31.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078255+0000", 2024-09-17T17:30:31.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.70988762599999999 2024-09-17T17:30:31.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078263+0000", 2024-09-17T17:30:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7430000000000002e-06 2024-09-17T17:30:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078349+0000", 2024-09-17T17:30:31.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6163999999999997e-05 2024-09-17T17:30:31.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078379+0000", 2024-09-17T17:30:31.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0278999999999999e-05 2024-09-17T17:30:31.782 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.782 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.782 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.783 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 75/74 ECSubWriteReply(tid=239, last_complete=75'130, committed=1, applied=1))", 2024-09-17T17:30:31.783 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:31.783 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 5.290243405, 2024-09-17T17:30:31.783 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.20952685800000001, 2024-09-17T17:30:31.783 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:31.784 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:31.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.785 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.785 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:31.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.68e-07 2024-09-17T17:30:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.787 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.787 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376368+0000", 2024-09-17T17:30:31.787 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:31.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.788 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585670+0000", 2024-09-17T17:30:31.788 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.20930257699999999 2024-09-17T17:30:31.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585686+0000", 2024-09-17T17:30:31.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5676e-05 2024-09-17T17:30:31.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585835+0000", 2024-09-17T17:30:31.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014933399999999999 2024-09-17T17:30:31.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.790 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585890+0000", 2024-09-17T17:30:31.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4969999999999997e-05 2024-09-17T17:30:31.791 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.791 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.791 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.792 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:796 3.16s0 3:6f166faa:::smithi10238019-38:head [copy-from ver 25] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:31.792 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:31.792 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.3056349899999999, 2024-09-17T17:30:31.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.020945305000000001, 2024-09-17T17:30:31.793 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:31.793 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.793 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.793 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 796 2024-09-17T17:30:31.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.794 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:31.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.795 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.795 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:31.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:31.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.01e-07 2024-09-17T17:30:31.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360975+0000", 2024-09-17T17:30:31.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5620000000000002e-06 2024-09-17T17:30:31.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360981+0000", 2024-09-17T17:30:31.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4989999999999999e-06 2024-09-17T17:30:31.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360988+0000", 2024-09-17T17:30:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3369999999999996e-06 2024-09-17T17:30:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361038+0000", 2024-09-17T17:30:31.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0141e-05 2024-09-17T17:30:31.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361130+0000", 2024-09-17T17:30:31.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1666e-05 2024-09-17T17:30:31.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:31.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.374988+0000", 2024-09-17T17:30:31.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01385871 2024-09-17T17:30:31.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.802 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:31.802 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.377231+0000", 2024-09-17T17:30:31.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022428610000000001 2024-09-17T17:30:31.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:31.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.381852+0000", 2024-09-17T17:30:31.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004620613 2024-09-17T17:30:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.381917+0000", 2024-09-17T17:30:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5514999999999999e-05 2024-09-17T17:30:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.805 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.805 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:797 3.16s0 3:6f166faa:::smithi10238019-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.305607918, 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016029564999999999, 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:31.807 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:31.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 797 2024-09-17T17:30:31.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.807 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:31.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4700000000000002e-07 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361002+0000", 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3929999999999998e-06 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361008+0000", 2024-09-17T17:30:31.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6470000000000001e-06 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361014+0000", 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4650000000000002e-06 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361241+0000", 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000227035 2024-09-17T17:30:31.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361277+0000", 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6492000000000002e-05 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.376753+0000", 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015475401999999999 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:31.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.376852+0000", 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9086999999999995e-05 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.377029+0000", 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017659699999999999 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.812 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.813 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:31.813 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:31.813 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:31.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:31.815 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:31.813+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:31.915 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:31.912+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:31.935 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:31.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:31.935 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:31.935 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:31.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:32.016 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:32.013+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:32.051 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:32.052 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:32.060 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:32.116 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:32.115+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:32.167 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.0s3 74/72 [PushOp(3:00c2d3b1:::smithi10238019-29:head, version: 73'62, data_included: [262144~184320], data_size: 184320, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:00c2d3b1:::smithi10238019-29:head@73'62, size: 1785856, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1785856, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.340156+0000", 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.8249040900000004, 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001270279, 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340156+0000", 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340156+0000", 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340157+0000", 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.046e-06 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340773+0000", 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061530799999999996 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340819+0000", 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6502000000000003e-05 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340840+0000", 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1157000000000001e-05 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.340934+0000", 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3820000000000006e-05 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341426+0000", 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049244600000000001 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.172 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.18s6 74/72 [PushOp(3:1f25649f:::smithi10238019-40:12, version: 70'26, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:1f25649f:::smithi10238019-40:12@70'26, size: 1687552, copy_subset: [], clone_subset: {}, snapset: 12={12: [11, 10]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.346198+0000", 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.8188625460000001, 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018691560000000001, 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346198+0000", 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346198+0000", 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346199+0000", 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8899999999999998e-07 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346380+0000", 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018149 2024-09-17T17:30:32.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346472+0000", 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2065999999999996e-05 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346489+0000", 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7059e-05 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.346557+0000", 2024-09-17T17:30:32.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7929999999999995e-05 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.348067+0000", 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015098220000000001 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.3s5 75/74 ECSubWrite(tid=93, reqid=client.4566.0:725, at_version=75'56, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.348168+0000", 2024-09-17T17:30:32.176 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8168918200000004, 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005632237, 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348168+0000", 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348168+0000", 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348169+0000", 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348348+0000", 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000179354 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348413+0000", 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4832000000000005e-05 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348430+0000", 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7370000000000001e-05 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348564+0000", 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000133666 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.348574+0000", 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.353761+0000", 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0051870270000000003 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.353801+0000", 2024-09-17T17:30:32.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9402e-05 2024-09-17T17:30:32.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.181 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.184 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/74 ECSubWrite(tid=163, reqid=osd.6.0:163, at_version=75'29, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.184 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.351029+0000", 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8140317819999998, 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0030867870000000001, 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351029+0000", 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351029+0000", 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351029+0000", 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6400000000000002e-07 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351051+0000", 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2109000000000001e-05 2024-09-17T17:30:32.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351086+0000", 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4826999999999997e-05 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351101+0000", 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5138e-05 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351162+0000", 2024-09-17T17:30:32.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0862000000000001e-05 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.351173+0000", 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0550000000000001e-05 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.354064+0000", 2024-09-17T17:30:32.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002891827 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.354115+0000", 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0909999999999999e-05 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.3s5 75/74 ECSubWrite(tid=94, reqid=osd.10.0:94, at_version=75'58, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.357921+0000", 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8071390130000005, 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043886100000000003, 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.357921+0000", 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.357921+0000", 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.357921+0000", 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.04e-07 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.357951+0000", 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9952000000000001e-05 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358047+0000", 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6074000000000002e-05 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358057+0000", 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6940000000000007e-06 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358140+0000", 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3117999999999999e-05 2024-09-17T17:30:32.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358149+0000", 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0599999999999997e-06 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362278+0000", 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004128858 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362310+0000", 2024-09-17T17:30:32.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1749999999999999e-05 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 75/74 ECSubWrite(tid=234, reqid=osd.13.0:234, at_version=75'125, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.360181+0000", 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8048796530000004, 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005382263, 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.195 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360181+0000", 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360181+0000", 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360181+0000", 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3099999999999998e-07 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360196+0000", 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.436e-05 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360220+0000", 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4833000000000001e-05 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360228+0000", 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8460000000000008e-06 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360282+0000", 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4132000000000003e-05 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360293+0000", 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0257e-05 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365532+0000", 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052392100000000002 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365563+0000", 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1093999999999997e-05 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 75/74 ECSubWrite(tid=235, reqid=osd.13.0:235, at_version=75'127, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8040617140000004, 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069013160000000002, 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1200000000000003e-07 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361014+0000", 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4908e-05 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361042+0000", 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8481000000000001e-05 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361050+0000", 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7109999999999995e-06 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361102+0000", 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1959999999999997e-05 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361112+0000", 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0345e-05 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.367865+0000", 2024-09-17T17:30:32.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067524300000000002 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.367900+0000", 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4968999999999997e-05 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 75/74 ECSubWrite(tid=236, reqid=osd.13.0:236, at_version=75'128, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:32.204 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8020826479999998, 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077947959999999997, 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362994+0000", 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5650000000000001e-05 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363018+0000", 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4300000000000001e-05 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363026+0000", 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4540000000000001e-06 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363078+0000", 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2086999999999998e-05 2024-09-17T17:30:32.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363088+0000", 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.005e-05 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370737+0000", 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076488509999999999 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370772+0000", 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5858999999999997e-05 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8012244620000004, 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095484720000000006, 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.209 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2799999999999996e-07 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363852+0000", 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515e-05 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363883+0000", 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1594999999999998e-05 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363893+0000", 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365558+0000", 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001665281 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365563+0000", 2024-09-17T17:30:32.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3410000000000002e-06 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373353+0000", 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077893490000000001 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373384+0000", 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1711999999999997e-05 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 75/74 ECSubWrite(tid=238, reqid=osd.13.0:235, at_version=0'0, trim_to=0'0, roll_forward_to=75'127))", 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.369909+0000", 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.7951509750000003, 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034895540000000002, 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.369909+0000", 2024-09-17T17:30:32.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.369909+0000", 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.369910+0000", 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.369923+0000", 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2646000000000001e-05 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.369956+0000", 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3293999999999997e-05 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.369966+0000", 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4830000000000005e-06 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370026+0000", 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0356999999999998e-05 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370037+0000", 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0713000000000001e-05 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373387+0000", 2024-09-17T17:30:32.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0033507150000000002 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373399+0000", 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1552000000000001e-05 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=104, reqid=client.4566.0:784, at_version=79'56, trim_to=0'0, roll_forward_to=78'54))", 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.353510+0000", 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.811549979, 2024-09-17T17:30:32.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054912479999999998, 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353510+0000", 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353510+0000", 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353511+0000", 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.085e-06 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353936+0000", 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00042424100000000001 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353973+0000", 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7188999999999999e-05 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.354020+0000", 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7574000000000002e-05 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.354096+0000", 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5597000000000001e-05 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.354103+0000", 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3710000000000002e-06 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.358933+0000", 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0048291669999999997 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359002+0000", 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9023999999999995e-05 2024-09-17T17:30:32.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=105, reqid=client.4566.0:785, at_version=79'57, trim_to=0'0, roll_forward_to=78'54))", 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.356799+0000", 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.808261017, 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057629370000000001, 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.356799+0000", 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.356799+0000", 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.356801+0000", 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.877e-06 2024-09-17T17:30:32.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357055+0000", 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025408199999999998 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357077+0000", 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1739999999999999e-05 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357085+0000", 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7710000000000006e-06 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357153+0000", 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8355000000000005e-05 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357162+0000", 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2439999999999999e-06 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362517+0000", 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053549890000000001 2024-09-17T17:30:32.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362562+0000", 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4879000000000002e-05 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=106, reqid=client.4566.0:786, at_version=79'58, trim_to=0'0, roll_forward_to=78'54))", 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.8041467899999999, 2024-09-17T17:30:32.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0061729920000000004, 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360915+0000", 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1769999999999999e-06 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361086+0000", 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017148 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361114+0000", 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7773999999999999e-05 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361124+0000", 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0380999999999999e-05 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361197+0000", 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3140999999999996e-05 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361218+0000", 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0197e-05 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367041+0000", 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005823249 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367086+0000", 2024-09-17T17:30:32.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5593000000000002e-05 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=107, reqid=client.4566.0:787, at_version=79'59, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.802177138, 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0070290120000000003, 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.233 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362884+0000", 2024-09-17T17:30:32.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.037e-06 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362968+0000", 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3614000000000003e-05 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363001+0000", 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3385e-05 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363013+0000", 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1761e-05 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363106+0000", 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2560999999999997e-05 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363117+0000", 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1457999999999999e-05 2024-09-17T17:30:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369881+0000", 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006764417 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369912+0000", 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0778999999999998e-05 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=108, reqid=client.4566.0:788, at_version=79'60, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.365091+0000", 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.7999696590000001, 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0048398180000000001, 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365091+0000", 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365091+0000", 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365092+0000", 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.547e-06 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365120+0000", 2024-09-17T17:30:32.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7696e-05 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365165+0000", 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5451999999999998e-05 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365176+0000", 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0213e-05 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365345+0000", 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016982 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365362+0000", 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6545000000000002e-05 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369916+0000", 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045541289999999996 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369930+0000", 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416000000000001e-05 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 79/74 ECSubWrite(tid=247, reqid=client.4566.0:796, at_version=79'135, trim_to=0'0, roll_forward_to=78'133))", 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.375033+0000", 2024-09-17T17:30:32.242 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.7900270740000002, 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004891533, 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375033+0000", 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375033+0000", 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375034+0000", 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9100000000000004e-07 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377605+0000", 2024-09-17T17:30:32.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025713770000000001 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377649+0000", 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3557000000000003e-05 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377657+0000", 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9419999999999995e-06 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377711+0000", 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4091000000000001e-05 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377717+0000", 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4309999999999999e-06 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.379879+0000", 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021618649999999998 2024-09-17T17:30:32.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.379925+0000", 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5779000000000003e-05 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 80/74 ECSubWrite(tid=249, reqid=osd.13.0:249, at_version=80'137, trim_to=0'0, roll_forward_to=79'135))", 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339567+0000", 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.82549324899999998, 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00171897, 2024-09-17T17:30:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339567+0000", 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339567+0000", 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339567+0000", 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1e-07 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339593+0000", 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5925e-05 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339641+0000", 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8093000000000002e-05 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339651+0000", 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.340068+0000", 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041697899999999999 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.340086+0000", 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7822999999999999e-05 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.341232+0000", 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011458320000000001 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.341286+0000", 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3879999999999999e-05 2024-09-17T17:30:32.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 80/74 ECSubWrite(tid=250, reqid=osd.13.0:249, at_version=0'0, trim_to=0'0, roll_forward_to=80'137))", 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.342179+0000", 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.82288167300000004, 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053858430000000004, 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342179+0000", 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342179+0000", 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342180+0000", 2024-09-17T17:30:32.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.023e-06 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342200+0000", 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.039e-05 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342244+0000", 2024-09-17T17:30:32.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3547000000000002e-05 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342253+0000", 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6290000000000005e-06 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342327+0000", 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4130999999999999e-05 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.342338+0000", 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0613e-05 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.347518+0000", 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0051802089999999999 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.347564+0000", 2024-09-17T17:30:32.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6301000000000003e-05 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 80/74 ECSubWrite(tid=111, reqid=osd.10.0:111, at_version=80'62, trim_to=0'0, roll_forward_to=79'60))", 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.346477+0000", 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.81858343499999997, 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005052567, 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346477+0000", 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346477+0000", 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346477+0000", 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0100000000000001e-07 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346521+0000", 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3906999999999998e-05 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346600+0000", 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8857999999999996e-05 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346610+0000", 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0368e-05 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346695+0000", 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4450000000000006e-05 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.346702+0000", 2024-09-17T17:30:32.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.401e-06 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.351482+0000", 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047794279999999996 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.351529+0000", 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7854e-05 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 80/74 ECSubWrite(tid=112, reqid=osd.10.0:111, at_version=0'0, trim_to=0'0, roll_forward_to=80'62))", 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.352299+0000", 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.81276155000000005, 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002301572, 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352299+0000", 2024-09-17T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352299+0000", 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352300+0000", 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5999999999999991e-07 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352325+0000", 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.561e-05 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352373+0000", 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7851e-05 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352384+0000", 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0642000000000001e-05 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352473+0000", 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8776999999999994e-05 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352487+0000", 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4146e-05 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.354558+0000", 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020717230000000001 2024-09-17T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.354600+0000", 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1863000000000002e-05 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:32.268 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:32.268 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:32.265+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:32.324 INFO:tasks.rados.rados.0.smithi102.stdout:303: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:32.324 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:32.326 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:32.328 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:298: done (5 left) 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:299: done (4 left) 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:300: done (3 left) 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:301: done (2 left) 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:303: done (1 left) 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:304: done (0 left) 2024-09-17T17:30:32.349 INFO:tasks.rados.rados.0.smithi102.stdout:305: snap_create 2024-09-17T17:30:32.369 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:32.367+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:32.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:32.421 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:32.421 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:32.421 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:32.421 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:32.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:32.469 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:32.467+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:32.483 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 2] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:30:32.483 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:32.483 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T17:30:32.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T17:30:32.512 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:32.513 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:32.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:32.536 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:32.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:32.537 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:32.537 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:32.537 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:32.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:32.556 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:32.570 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:32.568+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:32.578 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:32.601 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:32.602 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:32.623 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:32.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T17:30:32.644 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:32.653 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:32.653 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:32.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:32.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:32.671 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:32.668+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:32.688 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:32.772 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:32.772 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:32.772 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:32.772 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:32.772 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:32.770+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:32.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:32.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T17:30:32.873 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:32.872+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:32.887 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:32.888 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:32.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:32.927 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:32.928 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:32.955 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:32.955 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:32.974 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:32.972+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:32.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:32.978 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:33.000 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:33.000 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:33.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T17:30:33.010 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:33.010 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:33.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:33.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:33.022 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:33.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:33.020+0000 7f421ffff640 0 -- 172.21.15.7:0/2962743582 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f422021f130 msgr2=0x7f4220221520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:33.075 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:33.073+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:33.135 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:33.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-17T17:30:33.179 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:33.177+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:33.242 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:33.243 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:33.245 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:33.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:33.266 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:33.266 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:33.277 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:33.275+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:33.349 INFO:tasks.rados.rados.0.smithi102.stdout:305: done (0 left) 2024-09-17T17:30:33.349 INFO:tasks.rados.rados.0.smithi102.stdout:306: rollback oid 30 current snap is 26 2024-09-17T17:30:33.349 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 30 to 24 2024-09-17T17:30:33.349 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 9 current snap is 26 2024-09-17T17:30:33.350 INFO:tasks.rados.rados.0.smithi102.stdout:307: seq_num 136 ranges {819200=737280,1556480=770048,2326528=540672,2867200=245760} 2024-09-17T17:30:33.358 INFO:tasks.rados.rados.0.smithi102.stdout:307: writing smithi10238019-9 from 819200 to 1556480 tid 1 2024-09-17T17:30:33.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-17T17:30:33.363 INFO:tasks.rados.rados.0.smithi102.stdout:307: writing smithi10238019-9 from 1556480 to 2326528 tid 2 2024-09-17T17:30:33.366 INFO:tasks.rados.rados.0.smithi102.stdout:307: writing smithi10238019-9 from 2326528 to 2867200 tid 3 2024-09-17T17:30:33.368 INFO:tasks.rados.rados.0.smithi102.stdout:307: writing smithi10238019-9 from 2867200 to 3112960 tid 4 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:308: read oid 13 snap 25 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:308: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 28 current snap is 26 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:306: finishing rollback tid 0 to smithi10238019-30 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:306: finishing rollback tid 1 to smithi10238019-30 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:306: finishing rollback tid 2 to smithi10238019-30 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 30 v 68 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:307: finishing write tid 1 to smithi10238019-9 2024-09-17T17:30:33.369 INFO:tasks.rados.rados.0.smithi102.stdout:309: seq_num 137 ranges {0=753664,753664=786432,1540096=655360,2195456=114688} 2024-09-17T17:30:33.373 INFO:tasks.rados.rados.0.smithi102.stdout:309: writing smithi10238019-28 from 0 to 753664 tid 1 2024-09-17T17:30:33.377 INFO:tasks.rados.rados.0.smithi102.stdout:309: writing smithi10238019-28 from 753664 to 1540096 tid 2 2024-09-17T17:30:33.381 INFO:tasks.rados.rados.0.smithi102.stdout:309: writing smithi10238019-28 from 1540096 to 2195456 tid 3 2024-09-17T17:30:33.382 INFO:tasks.rados.rados.0.smithi102.stdout:309: writing smithi10238019-28 from 2195456 to 2310144 tid 4 2024-09-17T17:30:33.383 INFO:tasks.rados.rados.0.smithi102.stdout:306: done (3 left) 2024-09-17T17:30:33.383 INFO:tasks.rados.rados.0.smithi102.stdout:310: delete oid 42 current snap is 26 2024-09-17T17:30:33.383 INFO:tasks.rados.rados.0.smithi102.stdout:307: finishing write tid 2 to smithi10238019-9 2024-09-17T17:30:33.383 INFO:tasks.rados.rados.0.smithi102.stdout:307: finishing write tid 3 to smithi10238019-9 2024-09-17T17:30:33.383 INFO:tasks.rados.rados.0.smithi102.stdout:308: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:33.383 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:33.384 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:33.386 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:307: finishing write tid 4 to smithi10238019-9 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:307: finishing write tid 5 to smithi10238019-9 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:309: finishing write tid 1 to smithi10238019-28 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:307: finishing write tid 6 to smithi10238019-9 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:307: oid 9 updating version 0 to 54 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:307: oid 9 updating version 54 to 58 2024-09-17T17:30:33.389 INFO:tasks.rados.rados.0.smithi102.stdout:307: oid 9 version 58 is already newer than 57 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:307: oid 9 version 58 is already newer than 55 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:307: oid 9 version 58 is already newer than 56 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 58 (ObjNum 136 snap 26 seq_num 136) dirty exists 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:307: left oid 9 (ObjNum 136 snap 26 seq_num 136) 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:309: finishing write tid 2 to smithi10238019-28 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:307: done (3 left) 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:308: done (2 left) 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:310: done (1 left) 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:311: copy_from oid 24 from oid 30 current snap is 26 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:312: read oid 8 snap -1 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:312: expect (ObjNum 99 snap 13 seq_num 99) 2024-09-17T17:30:33.390 INFO:tasks.rados.rados.0.smithi102.stdout:313: delete oid 16 current snap is 26 2024-09-17T17:30:33.395 INFO:tasks.rados.rados.0.smithi102.stdout:313: done (3 left) 2024-09-17T17:30:33.395 INFO:tasks.rados.rados.0.smithi102.stdout:314: read oid 26 snap -1 2024-09-17T17:30:33.395 INFO:tasks.rados.rados.0.smithi102.stdout:314: expect (ObjNum 25 snap 0 seq_num 25) 2024-09-17T17:30:33.395 INFO:tasks.rados.rados.0.smithi102.stdout:315: snap_remove snap 17 2024-09-17T17:30:33.404 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:33.402+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:33.412 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:33.412 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:33.421 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:33.479 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:33.477+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:33.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-17T17:30:33.528 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:33.528 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:33.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:33.579 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:33.577+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:33.639 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:33.640 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:33.648 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:33.681 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:33.678+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:33.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 reset_purged_snaps_last 2024-09-17T17:30:33.751 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:33.751 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:33.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:33.803 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:33.801+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:33.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 reset_purged_snaps_last 2024-09-17T17:30:33.862 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:33.862 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:33.870 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:33.882 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:33.879+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:33.983 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:33.980+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:34.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 reset_purged_snaps_last 2024-09-17T17:30:34.013 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.013 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.021 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:34.088 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:34.085+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:34.124 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.124 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.133 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:34.185 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:34.182+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:34.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 reset_purged_snaps_last 2024-09-17T17:30:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.237 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.244 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:34.285 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:34.282+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:34.346 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.346 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.350 INFO:tasks.rados.rados.0.smithi102.stdout:315: done (4 left) 2024-09-17T17:30:34.350 INFO:tasks.rados.rados.0.smithi102.stdout:316: delete oid 37 current snap is 26 2024-09-17T17:30:34.350 INFO:tasks.rados.rados.0.smithi102.stdout:309: finishing write tid 3 to smithi10238019-28 2024-09-17T17:30:34.351 INFO:tasks.rados.rados.0.smithi102.stdout:309: finishing write tid 4 to smithi10238019-28 2024-09-17T17:30:34.351 INFO:tasks.rados.rados.0.smithi102.stdout:309: finishing write tid 5 to smithi10238019-28 2024-09-17T17:30:34.351 INFO:tasks.rados.rados.0.smithi102.stdout:311: finishing copy_from racing read to smithi10238019-24 2024-09-17T17:30:34.351 INFO:tasks.rados.rados.0.smithi102.stdout:312: expect (ObjNum 99 snap 13 seq_num 99) 2024-09-17T17:30:34.351 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:34.363 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:34.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:30:34.400 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:34.408 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:34.405+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:34.456 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 2760713 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: finishing write tid 6 to smithi10238019-28 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: oid 28 updating version 0 to 138 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: oid 28 updating version 138 to 142 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: oid 28 version 142 is already newer than 140 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: oid 28 version 142 is already newer than 141 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: oid 28 version 142 is already newer than 139 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 28 v 142 (ObjNum 137 snap 26 seq_num 137) dirty exists 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:309: left oid 28 (ObjNum 137 snap 26 seq_num 137) 2024-09-17T17:30:34.461 INFO:tasks.rados.rados.0.smithi102.stdout:314: expect (ObjNum 25 snap 0 seq_num 25) 2024-09-17T17:30:34.462 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 330941 2024-09-17T17:30:34.463 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:34.469 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 264208 2024-09-17T17:30:34.470 INFO:tasks.rados.rados.0.smithi102.stdout:311: finishing copy_from to smithi10238019-24 2024-09-17T17:30:34.470 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 24 v 68 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-09-17T17:30:34.471 INFO:tasks.rados.rados.0.smithi102.stdout:309: done (4 left) 2024-09-17T17:30:34.471 INFO:tasks.rados.rados.0.smithi102.stdout:311: done (3 left) 2024-09-17T17:30:34.471 INFO:tasks.rados.rados.0.smithi102.stdout:312: done (2 left) 2024-09-17T17:30:34.471 INFO:tasks.rados.rados.0.smithi102.stdout:314: done (1 left) 2024-09-17T17:30:34.471 INFO:tasks.rados.rados.0.smithi102.stdout:316: done (0 left) 2024-09-17T17:30:34.471 INFO:tasks.rados.rados.0.smithi102.stdout:317: snap_create 2024-09-17T17:30:34.471 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.471 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.479 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:34.521 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:34.518+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:34.540+0000 7f59477fe640 0 --2- 172.21.15.7:0/4293712884 >> v2:172.21.15.27:3300/0 conn(0x7f594815d370 0x7f59480cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T17:30:34.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 reset_purged_snaps_last 2024-09-17T17:30:34.588 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:34.586+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:34.623 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.623 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.631 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:34.689 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:34.686+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:34.733 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.733 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.742 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:34.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 reset_purged_snaps_last 2024-09-17T17:30:34.790 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:34.788+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:34.884 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:34.884 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:34.891 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:34.889+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:34.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:34.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 reset_purged_snaps_last 2024-09-17T17:30:34.991 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:34.989+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:35.002 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:35.002 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:35.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:35.093 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:35.091+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:35.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 reset_purged_snaps_last 2024-09-17T17:30:35.161 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:35.162 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:35.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:35.194 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:35.192+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:35.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:35.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-17T17:30:35.329 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:35.326+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:35.334 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:35.334 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:35.337 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:30:35.339 INFO:tasks.rados.rados.0.smithi102.stdout:317: done (0 left) 2024-09-17T17:30:35.339 INFO:tasks.rados.rados.0.smithi102.stdout:318: read oid 36 snap -1 2024-09-17T17:30:35.339 INFO:tasks.rados.rados.0.smithi102.stdout:318: expect (ObjNum 119 snap 19 seq_num 119) 2024-09-17T17:30:35.339 INFO:tasks.rados.rados.0.smithi102.stdout:319: delete oid 12 current snap is 27 2024-09-17T17:30:35.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:35.350 INFO:tasks.rados.rados.0.smithi102.stdout:319: done (1 left) 2024-09-17T17:30:35.350 INFO:tasks.rados.rados.0.smithi102.stdout:320: rmattr oid 46 current snap is 27 2024-09-17T17:30:35.350 INFO:tasks.rados.rados.0.smithi102.stdout:321: snap_remove snap 24 2024-09-17T17:30:35.396 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:35.393+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:35.427 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:35.450 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:35.450 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:35.471 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:35.494 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:35.495 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:35.496 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:35.493+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:35.512 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:35.513 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:35.515 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:35.515 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:35.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:35.537 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:35.537 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:35.558 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:35.580 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:35.580 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:35.596 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:35.594+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:35.600 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:35.622 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:35.622 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:35.644 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:35.645 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:35.646 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:35.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:35.666 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:35.666 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:35.687 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:35.697 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:35.695+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:35.708 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:35.709 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:35.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:35.706+0000 7f4948910640 0 -- 172.21.15.7:0/401305904 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f4944221670 msgr2=0x7f4944223aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:35.775 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.087694200000001, 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002820571, 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684890+0000", 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3499000000000001e-05 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685111+0000", 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022111 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685121+0000", 2024-09-17T17:30:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7070000000000004e-06 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685199+0000", 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8145999999999993e-05 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685209+0000", 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0138e-05 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687657+0000", 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024481260000000001 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687696+0000", 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.909e-05 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 50/35 ECSubWrite(tid=89, reqid=client.4566.0:401, at_version=50'26, trim_to=0'0, roll_forward_to=49'24))", 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.418888211999999, 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002465409, 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353682+0000", 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353683+0000", 2024-09-17T17:30:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2890000000000001e-06 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353717+0000", 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4183000000000001e-05 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353760+0000", 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.316e-05 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353773+0000", 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3037000000000001e-05 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353852+0000", 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8621000000000005e-05 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.353862+0000", 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0614e-05 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356113+0000", 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022509050000000001 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:01.356147+0000", 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999997e-05 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=94, reqid=client.4566.0:406, at_version=51'30, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.429358858999997, 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003277556, 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343212+0000", 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343225+0000", 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414999999999999e-05 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343249+0000", 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3983999999999999e-05 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343254+0000", 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5249999999999996e-06 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343307+0000", 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2167999999999999e-05 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343315+0000", 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8580000000000006e-06 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346453+0000", 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003137908 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346488+0000", 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5021999999999998e-05 2024-09-17T17:30:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 51/35 ECSubWrite(tid=62, reqid=osd.12.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.428016642999999, 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025408480000000001, 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344554+0000", 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8100000000000001e-07 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344567+0000", 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.254e-05 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344609+0000", 2024-09-17T17:30:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2549000000000002e-05 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344619+0000", 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6299999999999993e-06 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344683+0000", 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4090999999999993e-05 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344693+0000", 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0363000000000001e-05 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347044+0000", 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023509339999999998 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347094+0000", 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.986e-05 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.383052200999998, 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024942340000000001, 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389963+0000", 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044482000000000001 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389993+0000", 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.995e-05 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389999+0000", 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7830000000000004e-06 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390051+0000", 2024-09-17T17:30:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2475999999999997e-05 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390057+0000", 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7540000000000003e-06 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.391979+0000", 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001921835 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.392012+0000", 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2848000000000001e-05 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 56/35 ECSubWrite(tid=57, reqid=osd.10.0:57, at_version=56'41, trim_to=0'0, roll_forward_to=53'39))", 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.437306284000002, 2024-09-17T17:30:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036979209999999998, 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335264+0000", 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335298+0000", 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3837000000000001e-05 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335654+0000", 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035630699999999999 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335665+0000", 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1620999999999999e-05 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335751+0000", 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5692999999999994e-05 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335763+0000", 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140000000000001e-05 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338926+0000", 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031629399999999999 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338961+0000", 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5129999999999997e-05 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.434500570000001, 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003185701, 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338094+0000", 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4428000000000001e-05 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011644700000000001 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338219+0000", 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9020000000000005e-06 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338416+0000", 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196558 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338424+0000", 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.585e-06 2024-09-17T17:30:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341209+0000", 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027847050000000002 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341255+0000", 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.43445934, 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004124801, 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9600000000000001e-07 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338129+0000", 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8003e-05 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338243+0000", 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114367 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338249+0000", 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3160000000000002e-06 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338788+0000", 2024-09-17T17:30:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053876399999999995 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338797+0000", 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-06 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342196+0000", 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033993830000000002 2024-09-17T17:30:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342235+0000", 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8921999999999998e-05 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.434360406, 2024-09-17T17:30:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040538249999999996, 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.99e-07 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338233+0000", 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3623999999999999e-05 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338275+0000", 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2249000000000002e-05 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338284+0000", 2024-09-17T17:30:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2609999999999993e-06 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338436+0000", 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152032 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338448+0000", 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2469999999999999e-05 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342230+0000", 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037816899999999999 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342263+0000", 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3300000000000003e-05 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.434299795000001, 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029941270000000001, 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338289+0000", 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8340000000000001e-05 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338322+0000", 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3419e-05 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338332+0000", 2024-09-17T17:30:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0086e-05 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338873+0000", 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054120000000000004 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338882+0000", 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1329999999999999e-06 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341233+0000", 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023510699999999998 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1365999999999998e-05 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 57/35 ECSubWrite(tid=140, reqid=osd.6.0:140, at_version=57'15, trim_to=0'0, roll_forward_to=56'13))", 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.406041938000001, 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024870859999999999, 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366528+0000", 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T17:30:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366550+0000", 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1702e-05 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366615+0000", 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5030999999999995e-05 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366623+0000", 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366683+0000", 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9488999999999999e-05 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.366692+0000", 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5089999999999999e-06 2024-09-17T17:30:35.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.368974+0000", 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022823029999999998 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.369015+0000", 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0383000000000003e-05 2024-09-17T17:30:35.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.400194963000001, 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285746, 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:35.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372399+0000", 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4416000000000001e-05 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372479+0000", 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9556999999999997e-05 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372487+0000", 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7589999999999997e-06 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372561+0000", 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3807000000000007e-05 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372571+0000", 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0532e-05 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375622+0000", 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030507860000000002 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375661+0000", 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8634000000000003e-05 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.399465197000001, 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004179131, 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373125+0000", 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9626000000000001e-05 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373159+0000", 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4190999999999998e-05 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373168+0000", 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6999999999999997e-06 2024-09-17T17:30:35.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373237+0000", 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9306000000000004e-05 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373246+0000", 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377245+0000", 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039987390000000003 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377284+0000", 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8423999999999997e-05 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.399119078999998, 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048101309999999996, 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1800000000000005e-07 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373476+0000", 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4712999999999999e-05 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373535+0000", 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8536999999999999e-05 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373545+0000", 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0142e-05 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374163+0000", 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061771899999999997 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374172+0000", 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.081e-06 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378233+0000", 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040611509999999998 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378261+0000", 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000002e-05 2024-09-17T17:30:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.399068994, 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038195389999999998, 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373518+0000", 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373595+0000", 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6206999999999997e-05 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373601+0000", 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6039999999999998e-06 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373659+0000", 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7442000000000003e-05 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373668+0000", 2024-09-17T17:30:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2040000000000003e-06 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377300+0000", 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00363182 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377320+0000", 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0598000000000002e-05 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.396467469000001, 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034371530000000001, 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376103+0000", 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376128+0000", 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5109e-05 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376199+0000", 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1783e-05 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376209+0000", 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1710000000000008e-06 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376272+0000", 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749000000000002e-05 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376281+0000", 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379498+0000", 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003216605 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379539+0000", 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1587e-05 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.857 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 64/35 ECSubWrite(tid=108, reqid=client.4566.0:565, at_version=64'40, trim_to=0'0, roll_forward_to=60'38))", 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.435098076999999, 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002766442, 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337646+0000", 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017400099999999999 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337696+0000", 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0402999999999998e-05 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337712+0000", 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5441999999999999e-05 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337788+0000", 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6283000000000001e-05 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337842+0000", 2024-09-17T17:30:35.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3681999999999999e-05 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.340201+0000", 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023587859999999999 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.340238+0000", 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7404999999999998e-05 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 68/67 ECSubWrite(tid=75, reqid=client.4566.0:644, at_version=68'52, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.072622890000002, 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52365443700000003, 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699948+0000", 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999998e-07 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699964+0000", 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6784000000000001e-05 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222607+0000", 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52264301099999999 2024-09-17T17:30:35.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222621+0000", 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3923e-05 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222745+0000", 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012366800000000001 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222753+0000", 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6490000000000008e-06 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223566+0000", 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081285500000000002 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223601+0000", 2024-09-17T17:30:35.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5744000000000002e-05 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/74 ECSubWrite(tid=163, reqid=osd.6.0:163, at_version=75'29, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4215079179999996, 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.34608951199999999, 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351063+0000", 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9999999999999997e-07 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351085+0000", 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2129000000000001e-05 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696028+0000", 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.344942945 2024-09-17T17:30:35.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696043+0000", 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5353e-05 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696175+0000", 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131996 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696189+0000", 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3573e-05 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697107+0000", 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091841899999999996 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697151+0000", 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4397000000000002e-05 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.13s4 80/74 ECSubWrite(tid=111, reqid=osd.10.0:111, at_version=80'62, trim_to=0'0, roll_forward_to=79'60))", 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.4251493450000003, 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003050119, 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:35.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347421+0000", 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.03e-07 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347462+0000", 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0862999999999998e-05 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347534+0000", 2024-09-17T17:30:35.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2643000000000002e-05 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347550+0000", 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5633000000000001e-05 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347706+0000", 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015574400000000001 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347737+0000", 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1164000000000002e-05 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350445+0000", 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027080540000000001 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350470+0000", 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5715e-05 2024-09-17T17:30:35.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:35.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:35.878 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:35.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:35.880 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:35.878+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:35.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:35.933 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:35.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:35.954 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:35.979 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:35.978+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:36.030 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:36.030 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:36.030 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:36.030 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:36.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:36.081 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:36.080+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:36.146 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:36.146 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:36.146 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:36.146 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:36.147 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:36.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:36.182 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:36.180+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.576665759999997, 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:30:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.563129590999999, 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:30:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:30:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.560903732, 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:30:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:30:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:30:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:30:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.560479653000002, 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:30:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.560385351000001, 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:30:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:30:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.bs4 43/32 ECSubWrite(tid=92, reqid=osd.13.0:92, at_version=43'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-17T17:30:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.563445659999999, 2024-09-17T17:30:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019485469999999999, 2024-09-17T17:30:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697877+0000", 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697917+0000", 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9153999999999998e-05 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697948+0000", 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1538000000000002e-05 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697957+0000", 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698771+0000", 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081437600000000001 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.698781+0000", 2024-09-17T17:30:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9590000000000007e-06 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699776+0000", 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099482900000000003 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.699825+0000", 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9280000000000003e-05 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 51/32 ECSubWrite(tid=61, reqid=osd.3.0:61, at_version=51'39, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.921131205000002, 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00196635, 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340191+0000", 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340192+0000", 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340213+0000", 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.145e-05 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340244+0000", 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1063999999999999e-05 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340255+0000", 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1066e-05 2024-09-17T17:30:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8850000000000007e-05 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.340334+0000", 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0081999999999999e-05 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.342124+0000", 2024-09-17T17:30:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001789646 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.342157+0000", 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3788999999999998e-05 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=82, reqid=osd.3.0:82, at_version=57'61, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.890047497000001, 2024-09-17T17:30:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035036519999999999, 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-07 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371301+0000", 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6461e-05 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372066+0000", 2024-09-17T17:30:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076444900000000005 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372076+0000", 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372145+0000", 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9586999999999996e-05 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372155+0000", 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0054999999999999e-05 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374734+0000", 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790520000000001 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374778+0000", 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4042000000000003e-05 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=57'63, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.8879582, 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026238189999999999, 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373365+0000", 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0640000000000001e-06 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373399+0000", 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3757000000000002e-05 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373439+0000", 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0028999999999999e-05 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373469+0000", 2024-09-17T17:30:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0297000000000001e-05 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373543+0000", 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4202999999999994e-05 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373574+0000", 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0895999999999997e-05 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375923+0000", 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023489689999999998 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375988+0000", 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4603999999999993e-05 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:581 3.es0 3:7a426564:::smithi10238019-31:head [create,setxattr TahZbq^WXaahVeQfQsXqNXgcd (1963) in=25b,setxattr `Pj`q[kZqXoPYYj_`hNV^bohgce[Ull^lRXTctln (982) in=40b,setxattr ^gfWYcnTRmaWZQkrf`asZ\\XVPcRmRdRckrhUheci (10312) in=40b,setxattr hVm[Q^c]jmYfcedO_eoe[N_fohZ^nghUPgNQYY`h (15648) in=40b,setxattr \\jlYtrYeh_rthhUaqacc]`Uger\\]rPngbjPhbaWV (454) in=40b,setxattr nOgORnqOheegXnS]tSkTQTreXqqt`ojnVPsQbabf (771) in=40b,setxattr QR]c[o[]SToV\\goc\\pjtRRf`WYSngR[eQ`taQgcc (12488) in=40b,setxattr pY\\tNq\\a[n\\fajnSVYNng[\\kcqZ^VhqNmqRXUTlO (1121) in=40b,setxattr [laaiUa\\cY`kdXP\\SNRq[ig]rcanlRgnTtdfZi_W (1403) in=40b,setxattr aaf[PNUURnfSbT^fR_ge[lfbq]keQjYgrbO\\SbTa (1226) in=40b,setxattr SfaWR`YpWpbnn]sR]^eO[Wbi\\Se_TkrcQp[pcomR (4768) in=40b,setxattr pS^PVQeeOqs_S^YVlb^n\\h``m_Q\\prek^_qjcmgR (1121) in=40b,setxattr [aWfmed^jZheSjc^Xmb]TrYaU[[c]jb\\[UdNQngi (593) in=40b,setxattr kPO^emkeWSaQtnWaRVXk[hbN[TP^]ThcqpOfordV (2805) in=40b,setxattr PkhW`mV]l_O[nUSprX^tRPoRakfUNbOiRedpZb^n (421) in=40b,setxattr [`esP (1963) in=5b,setxattr SlhkjUdadks\\hNrmRohr_fl`itUjb_Vjia\\Vdqf` (10033) in=40b,setxattr TUWTXRdZcQc\\lOTbYmrWiirilafTQQrNdPoj_^sZ (3084) in=40b,setxattr ddbfaqshSg (1963) in=10b,setxattr nSgahmseaQO^hNUcamf\\m[SsQam^Uc``at``i`kU (1435) in=40b] snapc 11=[11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.899454995999999, 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047340350000000002, 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 581 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361869+0000", 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.623e-06 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362010+0000", 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014150499999999999 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362027+0000", 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6569e-05 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362040+0000", 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2767e-05 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362112+0000", 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2401e-05 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362122+0000", 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0013000000000001e-05 2024-09-17T17:30:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.363809+0000", 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016863329999999999 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.363887+0000", 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8323000000000002e-05 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.364760+0000", 2024-09-17T17:30:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087325800000000002 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.365703+0000", 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094332599999999997 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.366501+0000", 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000797326 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.366601+0000", 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100591 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:656 3.es0 3:7a426564:::smithi10238019-31:head [append 0~0] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.913805911000001, 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081233910000000006, 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 656 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347519+0000", 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6850000000000001e-06 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347531+0000", 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2339e-05 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347553+0000", 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2005999999999999e-05 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347564+0000", 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0944999999999999e-05 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347643+0000", 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8885000000000001e-05 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347652+0000", 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T17:30:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350793+0000", 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031410100000000001 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350899+0000", 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105824 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.351778+0000", 2024-09-17T17:30:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087886400000000003 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353482+0000", 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001703577 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355585+0000", 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021028700000000002 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355640+0000", 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5186999999999999e-05 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:657 3.es0 3:7a426564:::smithi10238019-31:head [rollback f] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.913726626000001, 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088160730000000007, 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 657 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7800000000000001e-07 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347600+0000", 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.501e-06 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347608+0000", 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5990000000000002e-06 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347615+0000", 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347721+0000", 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106421 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347727+0000", 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0349999999999998e-06 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.352452+0000", 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047242070000000002 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.352548+0000", 2024-09-17T17:30:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5958000000000005e-05 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353151+0000", 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060365899999999999 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354055+0000", 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090400699999999997 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356330+0000", 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022750029999999998 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356412+0000", 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1454999999999999e-05 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 69/32 ECSubWrite(tid=126, reqid=client.4566.0:661, at_version=69'95, trim_to=0'0, roll_forward_to=69'93))", 2024-09-17T17:30:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.367835+0000", 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.893486829, 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023874859999999999, 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.367835+0000", 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.367835+0000", 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.367836+0000", 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5899999999999996e-07 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368759+0000", 2024-09-17T17:30:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000922532 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368785+0000", 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6265999999999999e-05 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368790+0000", 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3020000000000002e-06 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368839+0000", 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9098000000000003e-05 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368847+0000", 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2389999999999996e-06 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.370196+0000", 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013490640000000001 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.370223+0000", 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7325999999999999e-05 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:726 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~606208] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.909719497999999, 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049411350000000001, 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 726 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351604+0000", 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6700000000000001e-06 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352206+0000", 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060191499999999996 2024-09-17T17:30:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352220+0000", 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.342e-05 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352227+0000", 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5989999999999996e-06 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352307+0000", 2024-09-17T17:30:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9568999999999997e-05 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352377+0000", 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9842e-05 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.353455+0000", 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001077801 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354825+0000", 2024-09-17T17:30:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001370509 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356461+0000", 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016354329999999999 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356544+0000", 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3376999999999998e-05 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:727 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~507904] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.90675304, 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027356989999999999, 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 727 2024-09-17T17:30:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354571+0000", 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.302e-06 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355037+0000", 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046691300000000001 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355070+0000", 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2064000000000003e-05 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355079+0000", 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8260000000000005e-06 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355121+0000", 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1813000000000001e-05 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355196+0000", 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4367000000000001e-05 2024-09-17T17:30:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355749+0000", 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055320300000000005 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356631+0000", 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00088253400000000003 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357264+0000", 2024-09-17T17:30:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063273500000000005 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357305+0000", 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0942000000000002e-05 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:728 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~573440] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.903515799000001, 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028748519999999998, 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 728 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357808+0000", 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3650000000000001e-06 2024-09-17T17:30:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358354+0000", 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054612300000000003 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358365+0000", 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1242e-05 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358371+0000", 2024-09-17T17:30:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5219999999999998e-06 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358404+0000", 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139000000000002e-05 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358466+0000", 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2359000000000004e-05 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358974+0000", 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050814800000000004 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.359801+0000", 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082624900000000004 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360638+0000", 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083704500000000002 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360681+0000", 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3659999999999999e-05 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:729 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~475136] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.900812856, 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034332189999999999, 2024-09-17T17:30:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 729 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360510+0000", 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0699999999999996e-07 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360948+0000", 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000437978 2024-09-17T17:30:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360962+0000", 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3478999999999999e-05 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360969+0000", 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4680000000000003e-06 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361023+0000", 2024-09-17T17:30:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3381999999999998e-05 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361088+0000", 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5579000000000004e-05 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361618+0000", 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052954599999999999 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362780+0000", 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001162787 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363897+0000", 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011169929999999999 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363943+0000", 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000001e-05 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:730 3.es0 3:74f6cb56:::smithi10238019-48:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.900335322, 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030870279999999999, 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 730 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360988+0000", 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.850000000000001e-07 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360995+0000", 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5120000000000002e-06 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361004+0000", 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-17T17:30:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361013+0000", 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4440000000000008e-06 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362000+0000", 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098763200000000001 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362055+0000", 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4580000000000003e-05 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362399+0000", 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034424899999999998 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363048+0000", 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00064926700000000003 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364011+0000", 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096274900000000005 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364074+0000", 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2903000000000004e-05 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:731 3.es0 3:74f6cb56:::smithi10238019-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.900297277, 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039696050000000002, 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 731 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361026+0000", 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361028+0000", 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.678e-06 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361035+0000", 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9829999999999999e-06 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361041+0000", 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-17T17:30:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362876+0000", 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001835267 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362939+0000", 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2508000000000005e-05 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364285+0000", 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013463780000000001 2024-09-17T17:30:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364367+0000", 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2318999999999996e-05 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364995+0000", 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000627182 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:7 3.es0 3:71a2b96a:::smithi10238019-23:head [assert-version v25,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9000745889999999, 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077714619999999998, 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361249+0000", 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.579e-06 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361265+0000", 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5812999999999999e-05 2024-09-17T17:30:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361290+0000", 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4666000000000001e-05 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361298+0000", 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1990000000000006e-06 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361353+0000", 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4898999999999997e-05 2024-09-17T17:30:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361425+0000", 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1644999999999996e-05 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.369019+0000", 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075946609999999999 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.377 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:36.378 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:36.375+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:36.379 INFO:tasks.rados.rados.0.smithi102.stdout:321: done (2 left) 2024-09-17T17:30:36.379 INFO:tasks.rados.rados.0.smithi102.stdout:322: copy_from oid 38 from oid 32 current snap is 27 2024-09-17T17:30:36.379 INFO:tasks.rados.rados.0.smithi102.stdout:318: expect (ObjNum 119 snap 19 seq_num 119) 2024-09-17T17:30:36.379 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1816191 2024-09-17T17:30:36.379 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1800 2024-09-17T17:30:36.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:36.385 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1880353 2024-09-17T17:30:36.387 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 61 (ObjNum 113 snap 18 seq_num 113) dirty exists 2024-09-17T17:30:36.387 INFO:tasks.rados.rados.0.smithi102.stdout:322: finishing copy_from racing read to smithi10238019-38 2024-09-17T17:30:36.387 INFO:tasks.rados.rados.0.smithi102.stdout:318: done (2 left) 2024-09-17T17:30:36.387 INFO:tasks.rados.rados.0.smithi102.stdout:320: done (1 left) 2024-09-17T17:30:36.388 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 45 current snap is 27 2024-09-17T17:30:36.388 INFO:tasks.rados.rados.0.smithi102.stdout:323: seq_num 142 ranges {0=442368,442368=720896,1163264=540672,1703936=688128} 2024-09-17T17:30:36.389 INFO:tasks.rados.rados.0.smithi102.stdout:323: writing smithi10238019-45 from 0 to 442368 tid 1 2024-09-17T17:30:36.397 INFO:tasks.rados.rados.0.smithi102.stdout:323: writing smithi10238019-45 from 442368 to 1163264 tid 2 2024-09-17T17:30:36.398 INFO:tasks.rados.rados.0.smithi102.stdout:323: writing smithi10238019-45 from 1163264 to 1703936 tid 3 2024-09-17T17:30:36.400 INFO:tasks.rados.rados.0.smithi102.stdout:323: writing smithi10238019-45 from 1703936 to 2392064 tid 4 2024-09-17T17:30:36.401 INFO:tasks.rados.rados.0.smithi102.stdout:324: snap_create 2024-09-17T17:30:36.401 INFO:tasks.rados.rados.0.smithi102.stdout:322: finishing copy_from to smithi10238019-38 2024-09-17T17:30:36.401 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 38 v 27 (ObjNum 125 snap 19 seq_num 125) dirty exists 2024-09-17T17:30:36.401 INFO:tasks.rados.rados.0.smithi102.stdout:323: finishing write tid 1 to smithi10238019-45 2024-09-17T17:30:36.402 INFO:tasks.rados.rados.0.smithi102.stdout:323: finishing write tid 2 to smithi10238019-45 2024-09-17T17:30:36.404 INFO:tasks.rados.rados.0.smithi102.stdout:323: finishing write tid 3 to smithi10238019-45 2024-09-17T17:30:36.407 INFO:tasks.rados.rados.0.smithi102.stdout:323: finishing write tid 4 to smithi10238019-45 2024-09-17T17:30:36.479 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:36.476+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:36.487 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:36.488 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:36.488 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:36.488 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:36.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:36.522 INFO:tasks.rados.rados.0.smithi102.stdout:323: finishing write tid 5 to smithi10238019-45 2024-09-17T17:30:36.523 INFO:tasks.rados.rados.0.smithi102.stdout:323: finishing write tid 6 to smithi10238019-45 2024-09-17T17:30:36.523 INFO:tasks.rados.rados.0.smithi102.stdout:323: oid 45 updating version 0 to 149 2024-09-17T17:30:36.523 INFO:tasks.rados.rados.0.smithi102.stdout:323: oid 45 updating version 149 to 152 2024-09-17T17:30:36.524 INFO:tasks.rados.rados.0.smithi102.stdout:323: oid 45 version 152 is already newer than 151 2024-09-17T17:30:36.524 INFO:tasks.rados.rados.0.smithi102.stdout:323: oid 45 updating version 152 to 153 2024-09-17T17:30:36.524 INFO:tasks.rados.rados.0.smithi102.stdout:323: oid 45 version 153 is already newer than 150 2024-09-17T17:30:36.524 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 45 v 153 (ObjNum 142 snap 27 seq_num 142) dirty exists 2024-09-17T17:30:36.524 INFO:tasks.rados.rados.0.smithi102.stdout:323: left oid 45 (ObjNum 142 snap 27 seq_num 142) 2024-09-17T17:30:36.581 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:36.579+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:36.649 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:36.649 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:36.649 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:36.650 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:36.650 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:36.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:36.681 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:36.679+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:36.767 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:36.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.081970138999999, 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:30:36.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:30:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:36.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:30:36.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.051016916000002, 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:36.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:30:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.018715272999998, 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:30:36.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:30:36.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.787 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.016790770999997, 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.792 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:410 3.9s0 3:93d1baed:::smithi10238019-5:head [copy-from ver 8] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.426894576999999, 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017989010999999999, 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 410 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:36.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338102+0000", 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5349999999999999e-06 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338107+0000", 2024-09-17T17:30:36.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000003e-06 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338121+0000", 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3743e-05 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338129+0000", 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4209999999999995e-06 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8885000000000003e-05 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338278+0000", 2024-09-17T17:30:36.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0192999999999995e-05 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.349495+0000", 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011217173 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350828+0000", 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013328700000000001 2024-09-17T17:30:36.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356035+0000", 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052068000000000001 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356090+0000", 2024-09-17T17:30:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4731000000000002e-05 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:411 3.9s0 3:93d1baed:::smithi10238019-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:36.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.426843578000003, 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012330081, 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 411 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338152+0000", 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338155+0000", 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0580000000000002e-06 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338164+0000", 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338170+0000", 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2809999999999997e-06 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338459+0000", 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028850300000000001 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338502+0000", 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3068000000000002e-05 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350266+0000", 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011764431000000001 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350370+0000", 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010380200000000001 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350482+0000", 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011173800000000001 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 52/35 ECSubWriteReply(tid=76, last_complete=52'24, committed=1, applied=1))", 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:36.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.355442953000001, 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.18030790999999999, 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409553+0000", 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409560+0000", 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589756+0000", 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.180196197 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589765+0000", 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589834+0000", 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9301000000000006e-05 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589860+0000", 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6058999999999999e-05 2024-09-17T17:30:36.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:477 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~802816] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.422930104999999, 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014281265, 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 477 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342065+0000", 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342067+0000", 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.906e-06 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343566+0000", 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001499521 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343581+0000", 2024-09-17T17:30:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4949e-05 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343590+0000", 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1529999999999997e-06 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343749+0000", 2024-09-17T17:30:36.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.343862+0000", 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011281399999999999 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.345103+0000", 2024-09-17T17:30:36.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001240718 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.346311+0000", 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012080210000000001 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356249+0000", 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099386809999999996 2024-09-17T17:30:36.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.356346+0000", 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7062e-05 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:479 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~131072] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:36.819 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.416917429000002, 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017167436000000001, 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 479 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348079+0000", 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.719e-06 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348181+0000", 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101258 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348196+0000", 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5423000000000001e-05 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348204+0000", 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348898+0000", 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069432499999999998 2024-09-17T17:30:36.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349042+0000", 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143848 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349752+0000", 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071013599999999999 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351054+0000", 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301618 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365180+0000", 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014125511 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365245+0000", 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5647000000000004e-05 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:480 3.0s0 3:00c2d3b1:::smithi10238019-29:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.416775060999999, 2024-09-17T17:30:36.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017175950999999998, 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 480 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.827 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348221+0000", 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348225+0000", 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969999999999998e-06 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348235+0000", 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312000000000001e-05 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348241+0000", 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340000000000002e-06 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350222+0000", 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019816209999999998 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350336+0000", 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113779 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350754+0000", 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041740400000000002 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351686+0000", 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093241600000000004 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365336+0000", 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013650254000000001 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365396+0000", 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9704000000000003e-05 2024-09-17T17:30:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:481 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.416743979, 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018899091, 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 481 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348252+0000", 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348255+0000", 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6720000000000002e-06 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348265+0000", 2024-09-17T17:30:36.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.132e-06 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348271+0000", 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351236+0000", 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029650309999999999 2024-09-17T17:30:36.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351333+0000", 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7449999999999997e-05 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365440+0000", 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106966 2024-09-17T17:30:36.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365519+0000", 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8703999999999997e-05 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.367150+0000", 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001631354 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 68/32 ECSubWrite(tid=191, reqid=client.4566.0:626, at_version=68'97, trim_to=0'0, roll_forward_to=68'94))", 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.264748998000002, 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018870181999999999, 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500247+0000", 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8299999999999996e-07 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500279+0000", 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2601000000000002e-05 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518513+0000", 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018233269 2024-09-17T17:30:36.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518520+0000", 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2590000000000003e-06 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518590+0000", 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:30:36.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518600+0000", 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7329999999999998e-06 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519071+0000", 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047090800000000003 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519116+0000", 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5521e-05 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.12s1 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.264307963, 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84719745400000002, 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:36.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500688+0000", 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3e-07 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500724+0000", 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-05 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346201+0000", 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84547703600000002 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346210+0000", 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346289+0000", 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8825999999999994e-05 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346300+0000", 2024-09-17T17:30:36.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1602e-05 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347848+0000", 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00154776 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347885+0000", 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6723000000000001e-05 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:646 3.0s0 3:07ceeeba:::smithi10238019-36:head [append 0~458752] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.337537+0000", 2024-09-17T17:30:36.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.427458101999999, 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013478109, 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 646 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.337537+0000", 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.337537+0000", 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.337540+0000", 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6989999999999999e-06 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.338020+0000", 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000480068 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.338043+0000", 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2796999999999999e-05 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.338053+0000", 2024-09-17T17:30:36.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0234000000000001e-05 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.338140+0000", 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6694999999999996e-05 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.338152+0000", 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1963e-05 2024-09-17T17:30:36.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.340634+0000", 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002482925 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.340863+0000", 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022893499999999999 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.341761+0000", 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089720000000000002 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.344885+0000", 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003124604 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350945+0000", 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060593380000000001 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.351015+0000", 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0650999999999998e-05 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:651 3.0s0 3:07ceeeba:::smithi10238019-36:head [setxattr _header (57) in=7b] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.417687082, 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.090354326999999998, 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 651 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347309+0000", 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347313+0000", 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7270000000000001e-06 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347320+0000", 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1019999999999999e-06 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347330+0000", 2024-09-17T17:30:36.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0552e-05 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353490+0000", 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061600309999999998 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353716+0000", 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000225987 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354187+0000", 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047076600000000001 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355021+0000", 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083427900000000005 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437590+0000", 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.082568902999999999 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437662+0000", 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2272000000000002e-05 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:652 3.0s0 3:07ceeeba:::smithi10238019-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.417655031999999, 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.091356190000000004, 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:36.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 652 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347341+0000", 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347344+0000", 2024-09-17T17:30:36.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0000000000000001e-06 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347352+0000", 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8250000000000005e-06 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347357+0000", 2024-09-17T17:30:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6489999999999996e-06 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354533+0000", 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007175781 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354625+0000", 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2064999999999994e-05 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438014+0000", 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.083388730999999994 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438102+0000", 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7861000000000005e-05 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438696+0000", 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059467799999999996 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 69/67 ECSubWriteReply(tid=135, last_complete=69'58, committed=1, applied=1))", 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.408866124999999, 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081843374999999996, 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356130+0000", 2024-09-17T17:30:36.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3600000000000003e-07 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356133+0000", 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.794e-06 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437473+0000", 2024-09-17T17:30:36.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081339488000000001 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437480+0000", 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6810000000000001e-06 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437533+0000", 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2958999999999999e-05 2024-09-17T17:30:36.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437972+0000", 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043981700000000001 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 73/72 ECSubReadReply(tid=170, attrs_read=0))", 2024-09-17T17:30:36.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.732062429999999, 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.308285174, 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.37e-07 2024-09-17T17:30:36.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.033151+0000", 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000217491 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337311+0000", 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.304160612 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337319+0000", 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4030000000000001e-06 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.340881+0000", 2024-09-17T17:30:36.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035619649999999998 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.341218+0000", 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033716600000000002 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 74/72 ECSubReadReply(tid=172, attrs_read=0))", 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.409918997, 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64532973699999996, 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355077+0000", 2024-09-17T17:30:36.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355304+0000", 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000086+0000", 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64478191900000004 2024-09-17T17:30:36.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000092+0000", 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1800000000000001e-06 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000163+0000", 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0883999999999994e-05 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000406+0000", 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024273599999999999 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPushReply(3.9s0 74/72 [PushReplyOp(3:96cbaefd:::smithi10238019-46:head)])", 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:36.891 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.752586356, 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32756439999999998, 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012412+0000", 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7250000000000002e-06 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337269+0000", 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32485740099999999 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337283+0000", 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3655999999999999e-05 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339351+0000", 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020684850000000001 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339973+0000", 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062179200000000003 2024-09-17T17:30:36.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:36.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:36.896 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:36.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:36.899 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:36.894+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:36.999 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:36.996+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:37.016 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:37.017 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:37.017 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:37.017 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:37.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:37.099 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:37.097+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:37.133 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:37.133 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:37.134 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:37.134 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:37.134 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:37.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:37.201 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:37.199+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:37.291 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:37.292 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:37.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:37.292 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:37.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.292 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:30:37.293 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:37.293 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 48.811532161000002, 2024-09-17T17:30:37.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:30:37.293 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.294 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:37.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:37.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:37.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:37.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:30:37.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:30:37.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:30:37.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:30:37.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:30:37.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:30:37.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:30:37.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:30:37.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:30:37.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:30:37.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:30:37.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:30:37.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:30:37.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:30:37.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.303 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.304 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:30:37.304 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:37.304 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 48.293265923, 2024-09-17T17:30:37.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:30:37.305 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.305 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.305 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:37.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:37.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:30:37.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:30:37.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:30:37.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:30:37.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:30:37.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:30:37.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:30:37.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:30:37.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:30:37.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:30:37.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:30:37.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:30:37.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.314 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:30:37.315 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:37.315 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.601548399999999, 2024-09-17T17:30:37.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:30:37.315 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.316 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:37.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:37.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:30:37.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:30:37.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:30:37.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:30:37.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:30:37.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:30:37.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:30:37.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:30:37.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:30:37.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:30:37.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:30:37.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:30:37.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.328 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:203 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~557056] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:37.328 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:37.329 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.584347489000002, 2024-09-17T17:30:37.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0069058180000000002, 2024-09-17T17:30:37.329 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.330 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.330 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.330 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 203 2024-09-17T17:30:37.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.330 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:37.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703968+0000", 2024-09-17T17:30:37.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703971+0000", 2024-09-17T17:30:37.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4849999999999999e-06 2024-09-17T17:30:37.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704582+0000", 2024-09-17T17:30:37.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00061129700000000001 2024-09-17T17:30:37.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704592+0000", 2024-09-17T17:30:37.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0094e-05 2024-09-17T17:30:37.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704598+0000", 2024-09-17T17:30:37.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6019999999999999e-06 2024-09-17T17:30:37.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704655+0000", 2024-09-17T17:30:37.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7107000000000001e-05 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704712+0000", 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7435000000000002e-05 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.705382+0000", 2024-09-17T17:30:37.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067020899999999995 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706343+0000", 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00096062299999999997 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710818+0000", 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044745519999999997 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.710874+0000", 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6413999999999999e-05 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.582105118000001, 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:30:37.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.345 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.581718215000002, 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:30:37.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:30:37.351 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.581684848000002, 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.352 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:30:37.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:30:37.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.577235377000001, 2024-09-17T17:30:37.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:30:37.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:367 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~589824] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.949624561999997, 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012183095999999999, 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.361 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 367 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:37.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338693+0000", 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.795e-06 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341609+0000", 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029156210000000002 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341623+0000", 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4583000000000001e-05 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341630+0000", 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2230000000000001e-06 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341691+0000", 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0878000000000002e-05 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341771+0000", 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9308999999999996e-05 2024-09-17T17:30:37.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343069+0000", 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012979420000000001 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348577+0000", 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0055080229999999999 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350781+0000", 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002204742 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350874+0000", 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2979999999999994e-05 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:368 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~573440] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:37.367 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.945984545000002, 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0088382370000000005, 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 368 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342332+0000", 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.08e-06 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342959+0000", 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062686600000000001 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343062+0000", 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010252 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343083+0000", 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1135e-05 2024-09-17T17:30:37.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344260+0000", 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011769969999999999 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344389+0000", 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012922399999999999 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345712+0000", 2024-09-17T17:30:37.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001323182 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348773+0000", 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030611649999999998 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351091+0000", 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023171590000000001 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351169+0000", 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8909e-05 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:369 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~196608] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.944882835999998, 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0079413130000000002, 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 369 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343433+0000", 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343434+0000", 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4699999999999999e-06 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343667+0000", 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023281800000000001 2024-09-17T17:30:37.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343680+0000", 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2524000000000001e-05 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343686+0000", 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.19e-06 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346596+0000", 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002909867 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.346803+0000", 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00020759199999999999 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348042+0000", 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001238636 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350047+0000", 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002004774 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351299+0000", 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001252364 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351374+0000", 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5078000000000002e-05 2024-09-17T17:30:37.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:370 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.944615093000003, 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0082819799999999996, 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.379 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 370 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:37.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3799999999999997e-07 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343710+0000", 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2570000000000007e-06 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343715+0000", 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6829999999999996e-06 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343721+0000", 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349122+0000", 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005400841 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349229+0000", 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107076 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349735+0000", 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050621599999999998 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351781+0000", 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020453279999999999 2024-09-17T17:30:37.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351908+0000", 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012790499999999999 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351983+0000", 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4154999999999997e-05 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:371 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.94458805, 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010270708, 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 371 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.385 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:37.386 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4799999999999999e-07 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343730+0000", 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4389999999999999e-06 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343735+0000", 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1919999999999998e-06 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.387 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343738+0000", 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1360000000000001e-06 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352473+0000", 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0087356429999999995 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352570+0000", 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6261999999999994e-05 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353998+0000", 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001428688 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.fs6 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.907937807, 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75327243099999996, 2024-09-17T17:30:37.389 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380379+0000", 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8100000000000002e-07 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380396+0000", 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7717e-05 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132975+0000", 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75257808000000004 2024-09-17T17:30:37.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132981+0000", 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2249999999999997e-06 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133046+0000", 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4944999999999996e-05 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.392 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133053+0000", 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133623+0000", 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057078100000000002 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133650+0000", 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6996e-05 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:495 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 28.630251678, 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.022462405000000001, 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 495 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.394 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.395 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658065+0000", 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2600000000000001e-07 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658072+0000", 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8430000000000004e-06 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658082+0000", 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.927e-06 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658089+0000", 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8909999999999998e-06 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660406+0000", 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023178890000000001 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660504+0000", 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7341000000000002e-05 2024-09-17T17:30:37.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661132+0000", 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062810899999999996 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661861+0000", 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072939299999999995 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:37.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680403+0000", 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018541588000000001 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680526+0000", 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012359799999999999 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/35 ECSubWriteReply(tid=37, last_complete=57'27, committed=1, applied=1))", 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:37.399 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 28.626484852000001, 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019253007999999999, 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.400 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661832+0000", 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661839+0000", 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7999999999999999e-06 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680112+0000", 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018272335000000001 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680126+0000", 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3867e-05 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680275+0000", 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000149741 2024-09-17T17:30:37.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.681084+0000", 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00080847500000000001 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:37.403 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.849451739999999, 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0071050749999999998, 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:37.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438864+0000", 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3699999999999999e-07 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438897+0000", 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3120999999999997e-05 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.405 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438953+0000", 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5955000000000002e-05 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.438970+0000", 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6263e-05 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441711+0000", 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027414589999999999 2024-09-17T17:30:37.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441722+0000", 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0797e-05 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.445920+0000", 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0041985929999999996 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.445969+0000", 2024-09-17T17:30:37.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8649999999999997e-05 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=196, reqid=client.4566.0:639, at_version=68'102, trim_to=0'0, roll_forward_to=68'97))", 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.762579415000001, 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.667795892, 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.408 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525737+0000", 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525756+0000", 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9026999999999999e-05 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192770+0000", 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.66701338300000002 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.410 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192793+0000", 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3203000000000001e-05 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192903+0000", 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109973 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192912+0000", 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9139999999999997e-06 2024-09-17T17:30:37.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193502+0000", 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058984700000000001 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193532+0000", 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0651999999999997e-05 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=197, reqid=client.4566.0:639, at_version=0'0, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.094406917000001, 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15173107799999999, 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5799999999999999e-07 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.414 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193938+0000", 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8988000000000001e-05 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344048+0000", 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15010903 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344059+0000", 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1148000000000001e-05 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.415 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344595+0000", 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00053675200000000004 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344608+0000", 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2211e-05 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345601+0000", 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00099356800000000001 2024-09-17T17:30:37.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345640+0000", 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8723000000000002e-05 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "replica_scrub(pg: 3.16s4,from:0'0,to:83'146,epoch:84/74,start:3:68000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.3666557799999999, 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.13087489099999999, 2024-09-17T17:30:37.417 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921661+0000", 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1230000000000001e-06 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921668+0000", 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5470000000000002e-06 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052265+0000", 2024-09-17T17:30:37.419 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.130597518 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052281+0000", 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5537e-05 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052444+0000", 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000163454 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.420 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052471+0000", 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6979999999999999e-05 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052535+0000", 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3732000000000006e-05 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.421 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:37.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:30:37.424 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:37.420+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:37.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:37.425 INFO:tasks.rados.rados.0.smithi102.stdout:322: done (2 left) 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:323: done (1 left) 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:324: done (0 left) 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:325: delete oid 10 current snap is 28 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:325: done (0 left) 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:326: delete oid 10 current snap is 28 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:326: done (0 left) 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 28 current snap is 28 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:327: seq_num 145 ranges {2310144=32768} 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:327: writing smithi10238019-28 from 2310144 to 2342912 tid 1 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:328: snap_create 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:327: finishing write tid 1 to smithi10238019-28 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:327: finishing write tid 2 to smithi10238019-28 2024-09-17T17:30:37.426 INFO:tasks.rados.rados.0.smithi102.stdout:327: finishing write tid 3 to smithi10238019-28 2024-09-17T17:30:37.427 INFO:tasks.rados.rados.0.smithi102.stdout:327: oid 28 updating version 0 to 155 2024-09-17T17:30:37.427 INFO:tasks.rados.rados.0.smithi102.stdout:327: oid 28 updating version 155 to 156 2024-09-17T17:30:37.427 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 28 v 156 (ObjNum 145 snap 28 seq_num 145) dirty exists 2024-09-17T17:30:37.427 INFO:tasks.rados.rados.0.smithi102.stdout:327: left oid 28 (ObjNum 145 snap 28 seq_num 145) 2024-09-17T17:30:37.525 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:37.523+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:37.536 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:37.536 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:37.536 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:37.536 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:37.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:37.627 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:37.625+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:37.692 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:37.692 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:37.692 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:37.692 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:37.692 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:37.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:37.730 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:37.728+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:37.809 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:37.809 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 48.814765987999998, 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:37.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:37.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.3s5 39/35 ECSubWrite(tid=18, reqid=client.4566.0:216, at_version=39'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 47.085829683, 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002528565, 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720702+0000", 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720703+0000", 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5300000000000004e-07 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721318+0000", 2024-09-17T17:30:37.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000615779 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721351+0000", 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2486e-05 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721360+0000", 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8729999999999996e-06 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721434+0000", 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4011999999999995e-05 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721445+0000", 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0903999999999999e-05 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723198+0000", 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001753641 2024-09-17T17:30:37.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.723231+0000", 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2217e-05 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.080692947999999, 2024-09-17T17:30:37.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:30:37.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:37.825 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.112665075999999, 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:30:37.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.112422965999997, 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:30:37.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:30:37.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:37.835 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 34.416888827000001, 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027457129999999999, 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2200000000000004e-07 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390408+0000", 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00076470399999999995 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390444+0000", 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5933000000000003e-05 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390453+0000", 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5120000000000008e-06 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390525+0000", 2024-09-17T17:30:37.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2849000000000007e-05 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390537+0000", 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1999e-05 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392350+0000", 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018122660000000001 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392389+0000", 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8828000000000002e-05 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/32 ECSubWrite(tid=110, reqid=client.4566.0:436, at_version=52'44, trim_to=0'0, roll_forward_to=52'42))", 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 34.405078013999997, 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002489495, 2024-09-17T17:30:37.840 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:37.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.401454+0000", 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3799999999999998e-07 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402030+0000", 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000576261 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402060+0000", 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9512000000000001e-05 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402068+0000", 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7970000000000001e-06 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402133+0000", 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5083e-05 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.402155+0000", 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1906999999999999e-05 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.403904+0000", 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017489510000000001 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.403943+0000", 2024-09-17T17:30:37.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9545999999999997e-05 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.468273519, 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024521180000000001, 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.845 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338258+0000", 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.22e-07 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338285+0000", 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.686e-05 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338389+0000", 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000103298 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338398+0000", 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5170000000000005e-06 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338479+0000", 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1052999999999998e-05 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.338489+0000", 2024-09-17T17:30:37.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6889999999999999e-06 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.340666+0000", 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021771529999999998 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.340710+0000", 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4226e-05 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.421931148999999, 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029370899999999998, 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:37.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384601+0000", 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8e-07 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385325+0000", 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072435200000000005 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385406+0000", 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0667999999999994e-05 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385415+0000", 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:30:37.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385509+0000", 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4137999999999999e-05 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385519+0000", 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0208e-05 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387496+0000", 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001976387 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387538+0000", 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1996999999999998e-05 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.421202987000001, 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029662, 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385362+0000", 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2403999999999997e-05 2024-09-17T17:30:37.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385471+0000", 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109485 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385479+0000", 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6750000000000002e-06 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386169+0000", 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068989099999999999 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386179+0000", 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0278000000000001e-05 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388245+0000", 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020665150000000001 2024-09-17T17:30:37.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388295+0000", 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9369000000000001e-05 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 59/32 ECSubWrite(tid=146, reqid=client.4566.0:504, at_version=59'61, trim_to=0'0, roll_forward_to=59'59))", 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:10.356814+0000", 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.449717399000001, 2024-09-17T17:30:37.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025928259999999999, 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.356814+0000", 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.356814+0000", 2024-09-17T17:30:37.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.356815+0000", 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9899999999999999e-07 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357476+0000", 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00066073 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357512+0000", 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5945999999999998e-05 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357522+0000", 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7790000000000007e-06 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357577+0000", 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5819999999999997e-05 2024-09-17T17:30:37.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.357588+0000", 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0118999999999999e-05 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.359376+0000", 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001788127 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.359407+0000", 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1405999999999997e-05 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 65/32 ECSubWrite(tid=171, reqid=client.4566.0:584, at_version=65'81, trim_to=0'0, roll_forward_to=62'79))", 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 21.439601604, 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023842849999999999, 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366930+0000", 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4799999999999999e-07 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.366961+0000", 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0667999999999998e-05 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367046+0000", 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5097000000000002e-05 2024-09-17T17:30:37.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367056+0000", 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.533e-06 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367138+0000", 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1819999999999999e-05 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.367149+0000", 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1131000000000001e-05 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.369264+0000", 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021152010000000001 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.369314+0000", 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0386999999999997e-05 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.868 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.367455886999998, 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029678690000000001, 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1899999999999998e-07 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439101+0000", 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4655e-05 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439183+0000", 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2472000000000006e-05 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439193+0000", 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0063e-05 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439257+0000", 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3775000000000005e-05 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439270+0000", 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2588e-05 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441995+0000", 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002725505 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442044+0000", 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8392e-05 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/67 ECSubWrite(tid=150, reqid=osd.6.0:150, at_version=68'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.357852088000001, 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023107700000000002, 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448680+0000", 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8999999999999995e-07 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448709+0000", 2024-09-17T17:30:37.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8249999999999999e-05 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448753+0000", 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4716999999999999e-05 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448765+0000", 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1959e-05 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448863+0000", 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7893000000000006e-05 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.448881+0000", 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7997000000000001e-05 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.450945+0000", 2024-09-17T17:30:37.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020640820000000001 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.450990+0000", 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4981999999999997e-05 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=188, reqid=client.4566.0:623, at_version=68'94, trim_to=0'0, roll_forward_to=68'92))", 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.492843+0000", 2024-09-17T17:30:37.878 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.31368818, 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002728776, 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492843+0000", 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492843+0000", 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492846+0000", 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.323e-06 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493606+0000", 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00076045400000000003 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493649+0000", 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2842000000000001e-05 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493659+0000", 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0179e-05 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493730+0000", 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1118000000000006e-05 2024-09-17T17:30:37.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493744+0000", 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3348e-05 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495531+0000", 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001786986 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495572+0000", 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1526000000000002e-05 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 69/32 ECSubWrite(tid=56, reqid=client.4566.0:656, at_version=69'42, trim_to=0'0, roll_forward_to=65'40))", 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.454670693000001, 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026116289999999999, 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351861+0000", 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.48e-07 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351878+0000", 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7079e-05 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351932+0000", 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4051000000000001e-05 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.351944+0000", 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1654999999999999e-05 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.352108+0000", 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016444800000000001 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.352129+0000", 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0545000000000001e-05 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.354429+0000", 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023003580000000002 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.354473+0000", 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3344999999999999e-05 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.887 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 70/67 ECSubWrite(tid=155, reqid=osd.6.0:155, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.461363215999999, 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00308939, 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345169+0000", 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:37.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345192+0000", 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3241e-05 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345239+0000", 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6607999999999998e-05 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345247+0000", 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5339999999999999e-06 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345711+0000", 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00046389400000000001 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345720+0000", 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0710000000000001e-06 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348210+0000", 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024892099999999999 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348258+0000", 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8208000000000003e-05 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=199, reqid=osd.11.0:199, at_version=70'104, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.461069209000001, 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002830782, 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345463+0000", 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.894 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345486+0000", 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3453999999999999e-05 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345559+0000", 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2760000000000001e-05 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345586+0000", 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7670000000000001e-05 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.895 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345650+0000", 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3139999999999995e-05 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345661+0000", 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1342000000000001e-05 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348252+0000", 2024-09-17T17:30:37.896 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025906829999999999 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348293+0000", 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.155e-05 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=200, reqid=osd.11.0:200, at_version=70'106, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.460487775000001, 2024-09-17T17:30:37.897 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027884030000000001, 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346045+0000", 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346081+0000", 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5942999999999998e-05 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.899 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346125+0000", 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3949999999999998e-05 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346134+0000", 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5019999999999998e-06 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346909+0000", 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077504099999999997 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346921+0000", 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1598e-05 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348789+0000", 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001868268 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348832+0000", 2024-09-17T17:30:37.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3059000000000002e-05 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 76/32 ECSubWrite(tid=67, reqid=client.4566.0:740, at_version=76'57, trim_to=0'0, roll_forward_to=73'55))", 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:27.364684+0000", 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.441847137, 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023389719999999999, 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:37.902 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.364684+0000", 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.364684+0000", 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:37.903 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.364685+0000", 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7300000000000002e-07 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365172+0000", 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048696299999999998 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365208+0000", 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6180999999999998e-05 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365217+0000", 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.799e-06 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365286+0000", 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9776000000000004e-05 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.365297+0000", 2024-09-17T17:30:37.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1018e-05 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.366991+0000", 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0016936169999999999 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:37.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:27.367023+0000", 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2345000000000003e-05 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:37.907 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:37.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:37.909 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:37.906+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:37.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:38.010 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:38.008+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:38.016 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:38.016 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:38.016 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:38.016 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:38.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:38.105 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:38.111 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:38.108+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:38.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:38.128 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:38.132 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:38.132 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:38.132 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:38.132 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:38.132 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:38.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:38.151 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:38.151 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:38.173 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:38.173 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:38.194 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:38.212 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:38.209+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:38.215 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:38.237 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:38.259 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:38.280 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:38.280 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:38.290 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:38.290 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:38.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.594125724000001, 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026508170000000002, 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694060+0000", 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694061+0000", 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5099999999999998e-07 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694087+0000", 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5571999999999998e-05 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694133+0000", 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5583e-05 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694142+0000", 2024-09-17T17:30:38.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6500000000000008e-06 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694227+0000", 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5159000000000002e-05 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694246+0000", 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8723e-05 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696667+0000", 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024210709999999999 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696711+0000", 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4107999999999998e-05 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:38.295 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 34.898547946000001, 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026689740000000002, 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389638+0000", 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389639+0000", 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.06e-07 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390292+0000", 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00065341900000000003 2024-09-17T17:30:38.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390322+0000", 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.921e-05 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390328+0000", 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5599999999999999e-06 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390385+0000", 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6484999999999999e-05 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390396+0000", 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1409e-05 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392271+0000", 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001875383 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392307+0000", 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5802e-05 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.948962261999998, 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032487919999999999, 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339225+0000", 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6799999999999999e-07 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339248+0000", 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3654000000000001e-05 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339288+0000", 2024-09-17T17:30:38.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9734999999999998e-05 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339297+0000", 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339672+0000", 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037528099999999999 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339682+0000", 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0168e-05 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342436+0000", 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027540889999999999 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342473+0000", 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6331999999999997e-05 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.948784162999999, 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030245340000000002, 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0100000000000001e-07 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339459+0000", 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6342999999999999e-05 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339692+0000", 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023318199999999999 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339701+0000", 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7469999999999994e-06 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339766+0000", 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5277999999999994e-05 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339776+0000", 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8439999999999993e-06 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342401+0000", 2024-09-17T17:30:38.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026253349999999999 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342426+0000", 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5503999999999998e-05 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=50, reqid=osd.7.0:49, at_version=0'0, trim_to=0'0, roll_forward_to=56'40))", 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339764+0000", 2024-09-17T17:30:38.309 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.948421798999998, 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026235939999999999, 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339764+0000", 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339764+0000", 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339765+0000", 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.68e-07 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339774+0000", 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3239999999999992e-06 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339828+0000", 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3384999999999998e-05 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339836+0000", 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0530000000000007e-06 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339918+0000", 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1786999999999995e-05 2024-09-17T17:30:38.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339953+0000", 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5077000000000003e-05 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342341+0000", 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023879510000000001 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342388+0000", 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7348999999999997e-05 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=88, reqid=client.4566.0:477, at_version=56'35, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353197+0000", 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.934989158, 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028115079999999999, 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.314 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353197+0000", 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353197+0000", 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353198+0000", 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1999999999999998e-07 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353639+0000", 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044126499999999998 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353970+0000", 2024-09-17T17:30:38.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00033084299999999999 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353980+0000", 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7880000000000001e-06 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354058+0000", 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8360000000000002e-05 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354069+0000", 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0899e-05 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355985+0000", 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019164970000000001 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356008+0000", 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3036000000000001e-05 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=90, reqid=client.4566.0:479, at_version=56'37, trim_to=0'0, roll_forward_to=56'32))", 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.934367307999999, 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026104409999999998, 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353819+0000", 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3000000000000001e-07 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353893+0000", 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4086999999999997e-05 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354045+0000", 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00015175099999999999 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354051+0000", 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8980000000000001e-06 2024-09-17T17:30:38.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355384+0000", 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013329189999999999 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355390+0000", 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0240000000000003e-06 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356391+0000", 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010013159999999999 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356429+0000", 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7916000000000001e-05 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=91, reqid=client.4566.0:480, at_version=56'38, trim_to=0'0, roll_forward_to=56'32))", 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353897+0000", 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.934288922, 2024-09-17T17:30:38.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027461170000000002, 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353897+0000", 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353897+0000", 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353898+0000", 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.63e-07 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353909+0000", 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1810999999999999e-05 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354065+0000", 2024-09-17T17:30:38.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00015532 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354072+0000", 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6290000000000001e-06 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355801+0000", 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0017290529999999999 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.355811+0000", 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8679999999999994e-06 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356611+0000", 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00079965099999999996 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356643+0000", 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2322e-05 2024-09-17T17:30:38.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=92, reqid=osd.12.0:92, at_version=56'40, trim_to=0'0, roll_forward_to=56'34))", 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.93427466, 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003074832, 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353912+0000", 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353937+0000", 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5015e-05 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354085+0000", 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014835900000000001 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354091+0000", 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8220000000000003e-06 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356082+0000", 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019913650000000001 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356088+0000", 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1190000000000002e-06 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356953+0000", 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00086492399999999999 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356986+0000", 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2994999999999998e-05 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.915733161999999, 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030867720000000002, 2024-09-17T17:30:38.332 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:38.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:38.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2800000000000001e-07 2024-09-17T17:30:38.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372483+0000", 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9967999999999998e-05 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372565+0000", 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2091999999999999e-05 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372574+0000", 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8869999999999998e-06 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372659+0000", 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5358999999999994e-05 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372674+0000", 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4827000000000001e-05 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375499+0000", 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028244870000000001 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375540+0000", 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1023999999999999e-05 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.337 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.914482489000001, 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033844539999999998, 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373730+0000", 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5689000000000001e-05 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373771+0000", 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1208000000000003e-05 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374141+0000", 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037011200000000002 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374273+0000", 2024-09-17T17:30:38.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013144900000000001 2024-09-17T17:30:38.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374283+0000", 2024-09-17T17:30:38.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0147e-05 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377050+0000", 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002767346 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377088+0000", 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7713000000000003e-05 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "pg_update_log_missing(3.cs6 epoch 59/32 rep_tid 142 entries 59'57 (0'0) error 3:3116c96c:::smithi10238019-11:head by client.4566.0:501 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 57'54)", 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.949685275, 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027076629999999999, 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:38.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338501+0000", 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.03e-07 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.338508+0000", 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1520000000000003e-06 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.339188+0000", 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068008399999999996 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.339207+0000", 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8369999999999999e-05 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.339358+0000", 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000151144 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:10.341209+0000", 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00185071 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 64/35 ECSubWrite(tid=108, reqid=client.4566.0:565, at_version=64'40, trim_to=0'0, roll_forward_to=60'38))", 2024-09-17T17:30:38.348 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.950592168, 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027638189999999998, 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337594+0000", 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.12e-07 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337844+0000", 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00024958599999999999 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337881+0000", 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7255999999999997e-05 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337892+0000", 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1274000000000001e-05 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.337984+0000", 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1729000000000002e-05 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.338000+0000", 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6157000000000001e-05 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.340311+0000", 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002311006 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:15.340358+0000", 2024-09-17T17:30:38.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6699000000000001e-05 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.847586242999999, 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003708654, 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:38.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5499999999999999e-07 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440633+0000", 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2728999999999998e-05 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440769+0000", 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013610200000000001 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440779+0000", 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9410000000000002e-06 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442220+0000", 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0014405760000000001 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442231+0000", 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1847e-05 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444268+0000", 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020363299999999998 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444309+0000", 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0774e-05 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.357 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 68/67 ECSubWrite(tid=122, reqid=client.4566.0:616, at_version=68'46, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.844113180000001, 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030735390000000001, 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444074+0000", 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1200000000000002e-07 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444177+0000", 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010335700000000001 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444221+0000", 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4370999999999999e-05 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444232+0000", 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0178e-05 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444294+0000", 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2204000000000003e-05 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444303+0000", 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9630000000000004e-06 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447110+0000", 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002806809 2024-09-17T17:30:38.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447146+0000", 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6944999999999999e-05 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/32 ECSubWrite(tid=188, reqid=client.4566.0:623, at_version=68'94, trim_to=0'0, roll_forward_to=68'92))", 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.492816+0000", 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.795370394999999, 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027185249999999998, 2024-09-17T17:30:38.362 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492816+0000", 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492816+0000", 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.492827+0000", 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.129e-05 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493571+0000", 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000743919 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493603+0000", 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1671000000000002e-05 2024-09-17T17:30:38.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493609+0000", 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.246e-06 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493676+0000", 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6816999999999994e-05 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.493686+0000", 2024-09-17T17:30:38.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0661e-05 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495499+0000", 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001812676 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.495534+0000", 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5244999999999999e-05 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.366 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=157, to_read={3:00c2d3b1:::smithi10238019-29:head=0,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 13.472903811, 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30626450500000002, 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:38.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815283+0000", 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9299999999999998e-07 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815290+0000", 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8709999999999999e-06 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121092+0000", 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30580278100000002 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121102+0000", 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5170000000000005e-06 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121186+0000", 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4229999999999998e-05 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121547+0000", 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00036061300000000003 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.370 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=171, to_read={3:00c2d3b1:::smithi10238019-29:head=262144,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 13.164066318, 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.21319038800000001, 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.61e-07 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124129+0000", 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5269999999999998e-06 2024-09-17T17:30:38.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336658+0000", 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.212529417 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336671+0000", 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2658e-05 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336735+0000", 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4203999999999997e-05 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.337310+0000", 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057492100000000005 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 85/76 ECSubWrite(tid=193, reqid=osd.12.0:193, at_version=85'63, trim_to=0'0, roll_forward_to=84'61))", 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.345245+0000", 2024-09-17T17:30:38.374 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.942940975, 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025676010000000001, 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345245+0000", 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345245+0000", 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345245+0000", 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4999999999999999e-07 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345279+0000", 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3235999999999998e-05 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345560+0000", 2024-09-17T17:30:38.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000281809 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345575+0000", 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4456e-05 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345686+0000", 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000111444 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.345699+0000", 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2300000000000001e-05 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.347766+0000", 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020677650000000001 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.347813+0000", 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6440999999999998e-05 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 85/32 ECSubWrite(tid=83, reqid=osd.7.0:83, at_version=85'73, trim_to=0'0, roll_forward_to=83'71))", 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 1.941642818, 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028233020000000002, 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:38.379 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346544+0000", 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3099999999999999e-07 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346574+0000", 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0852999999999998e-05 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346642+0000", 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7183999999999999e-05 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346653+0000", 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1076e-05 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346728+0000", 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5459999999999999e-05 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346741+0000", 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2894999999999999e-05 2024-09-17T17:30:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.349319+0000", 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00257771 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.349367+0000", 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7793000000000001e-05 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:38.383 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:38.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:38.385 INFO:tasks.rados.rados.0.smithi102.stdout:327: done (1 left) 2024-09-17T17:30:38.385 INFO:tasks.rados.rados.0.smithi102.stdout:328: done (0 left) 2024-09-17T17:30:38.385 INFO:tasks.rados.rados.0.smithi102.stdout:329: rmattr oid 29 current snap is 29 2024-09-17T17:30:38.385 INFO:tasks.rados.rados.0.smithi102.stdout:329: done (0 left) 2024-09-17T17:30:38.385 INFO:tasks.rados.rados.0.smithi102.stdout:330: delete oid 39 current snap is 29 2024-09-17T17:30:38.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:38.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:38.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:38.387 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:38.387 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:30:38.387 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:38.388 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:38.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:38.389 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:38.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:38.367+0000 7fefe6d9d640 0 -- 172.21.15.7:0/310505446 >> [v2:172.21.15.26:6800/1983765258,v1:172.21.15.26:6801/1983765258] conn(0x7fefe821ca50 msgr2=0x7fefe821ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-17T17:30:38.390 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:38.382+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:38.486 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:38.483+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:38.499 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:38.499 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:38.500 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:38.500 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:38.509 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:38.587 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:38.584+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:38.592 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:38.614 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:38.615 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:38.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:38.636 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:38.688 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:38.686+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:38.730 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:38.731 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:38.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:38.789 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:38.787+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:38.861 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:38.862 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:38.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:38.889 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:38.887+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:38.978 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:38.978 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:38.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:38.990 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:38.988+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:39.091 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:39.089+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:39.094 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.094 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:39.192 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:39.189+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:39.209 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.210 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:39.293 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:39.291+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:39.357 INFO:tasks.rados.rados.0.smithi102.stdout:330: done (0 left) 2024-09-17T17:30:39.358 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 29 current snap is 29 2024-09-17T17:30:39.358 INFO:tasks.rados.rados.0.smithi102.stdout:331: seq_num 147 ranges {1785856=65536} 2024-09-17T17:30:39.368 INFO:tasks.rados.rados.0.smithi102.stdout:331: writing smithi10238019-29 from 1785856 to 1851392 tid 1 2024-09-17T17:30:39.368 INFO:tasks.rados.rados.0.smithi102.stdout:332: read oid 41 snap -1 2024-09-17T17:30:39.368 INFO:tasks.rados.rados.0.smithi102.stdout:332: expect (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:30:39.368 INFO:tasks.rados.rados.0.smithi102.stdout:333: setattr oid 7 current snap is 29 2024-09-17T17:30:39.369 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.369 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.371 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 24 current snap is 29 2024-09-17T17:30:39.371 INFO:tasks.rados.rados.0.smithi102.stdout:334: seq_num 148 ranges {2228224=81920} 2024-09-17T17:30:39.381 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:39.385 INFO:tasks.rados.rados.0.smithi102.stdout:334: writing smithi10238019-24 from 2228224 to 2310144 tid 1 2024-09-17T17:30:39.385 INFO:tasks.rados.rados.0.smithi102.stdout:335: rmattr oid 17 current snap is 29 2024-09-17T17:30:39.385 INFO:tasks.rados.rados.0.smithi102.stdout:331: finishing write tid 1 to smithi10238019-29 2024-09-17T17:30:39.385 INFO:tasks.rados.rados.0.smithi102.stdout:332: expect (ObjNum 69 snap 4 seq_num 69) 2024-09-17T17:30:39.385 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:39.393 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:39.391+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:39.414 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1595905 2024-09-17T17:30:39.423 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1285596 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 7 v 86 (ObjNum 131 snap 22 seq_num 131) dirty exists 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:334: finishing write tid 1 to smithi10238019-24 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:334: finishing write tid 2 to smithi10238019-24 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 17 v 75 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:334: finishing write tid 3 to smithi10238019-24 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:334: oid 24 updating version 0 to 69 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:334: oid 24 updating version 69 to 70 2024-09-17T17:30:39.431 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 24 v 70 (ObjNum 148 snap 29 seq_num 148) dirty exists 2024-09-17T17:30:39.432 INFO:tasks.rados.rados.0.smithi102.stdout:334: left oid 24 (ObjNum 148 snap 29 seq_num 148) 2024-09-17T17:30:39.432 INFO:tasks.rados.rados.0.smithi102.stdout:332: done (4 left) 2024-09-17T17:30:39.432 INFO:tasks.rados.rados.0.smithi102.stdout:333: done (3 left) 2024-09-17T17:30:39.432 INFO:tasks.rados.rados.0.smithi102.stdout:334: done (2 left) 2024-09-17T17:30:39.432 INFO:tasks.rados.rados.0.smithi102.stdout:335: done (1 left) 2024-09-17T17:30:39.432 INFO:tasks.rados.rados.0.smithi102.stdout:336: delete oid 35 current snap is 29 2024-09-17T17:30:39.433 INFO:tasks.rados.rados.0.smithi102.stdout:336: done (1 left) 2024-09-17T17:30:39.433 INFO:tasks.rados.rados.0.smithi102.stdout:337: read oid 10 snap -1 2024-09-17T17:30:39.434 INFO:tasks.rados.rados.0.smithi102.stdout:337: expect deleted 2024-09-17T17:30:39.434 INFO:tasks.rados.rados.0.smithi102.stdout:338: rollback oid 9 current snap is 29 2024-09-17T17:30:39.434 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 9 to 19 2024-09-17T17:30:39.434 INFO:tasks.rados.rados.0.smithi102.stdout:339: delete oid 34 current snap is 29 2024-09-17T17:30:39.437 INFO:tasks.rados.rados.0.smithi102.stdout:338: finishing rollback tid 0 to smithi10238019-9 2024-09-17T17:30:39.437 INFO:tasks.rados.rados.0.smithi102.stdout:337: done (3 left) 2024-09-17T17:30:39.437 INFO:tasks.rados.rados.0.smithi102.stdout:339: done (2 left) 2024-09-17T17:30:39.437 INFO:tasks.rados.rados.0.smithi102.stdout:340: rollback oid 16 current snap is 29 2024-09-17T17:30:39.437 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 16 to 21 2024-09-17T17:30:39.437 INFO:tasks.rados.rados.0.smithi102.stdout:341: snap_remove snap 27 2024-09-17T17:30:39.490 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.490 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.495 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:39.493+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:39.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:39.596 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:39.594+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:39.606 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.606 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.615 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:39.696 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:39.694+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:39.721 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.721 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.730 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:39.797 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:39.795+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:39.837 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.837 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:39.898 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:39.896+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:39.993 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:39.993 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:39.999 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:39.997+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:40.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:40.101 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:40.099+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:40.148 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.148 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:40.201 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:40.199+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:40.264 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.264 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.273 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:40.302 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:40.300+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:341: done (3 left) 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:342: read oid 42 snap -1 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:342: expect deleted 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:343: delete oid 13 current snap is 29 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:338: finishing rollback tid 1 to smithi10238019-9 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:340: finishing rollback tid 1 to smithi10238019-16 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:340: finishing rollback tid 2 to smithi10238019-16 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 16 v 88 (ObjNum 95 snap 11 seq_num 95) dirty exists 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:338: finishing rollback tid 2 to smithi10238019-9 2024-09-17T17:30:40.341 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 68 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-17T17:30:40.342 INFO:tasks.rados.rados.0.smithi102.stdout:331: finishing write tid 2 to smithi10238019-29 2024-09-17T17:30:40.355 INFO:tasks.rados.rados.0.smithi102.stdout:338: done (4 left) 2024-09-17T17:30:40.355 INFO:tasks.rados.rados.0.smithi102.stdout:340: done (3 left) 2024-09-17T17:30:40.355 INFO:tasks.rados.rados.0.smithi102.stdout:342: done (2 left) 2024-09-17T17:30:40.355 INFO:tasks.rados.rados.0.smithi102.stdout:343: done (1 left) 2024-09-17T17:30:40.356 INFO:tasks.rados.rados.0.smithi102.stdout:344: snap_create 2024-09-17T17:30:40.374 INFO:tasks.rados.rados.0.smithi102.stdout:331: finishing write tid 3 to smithi10238019-29 2024-09-17T17:30:40.374 INFO:tasks.rados.rados.0.smithi102.stdout:331: oid 29 updating version 0 to 69 2024-09-17T17:30:40.375 INFO:tasks.rados.rados.0.smithi102.stdout:331: oid 29 version 69 is already newer than 68 2024-09-17T17:30:40.375 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 29 v 69 (ObjNum 147 snap 29 seq_num 147) dirty exists 2024-09-17T17:30:40.375 INFO:tasks.rados.rados.0.smithi102.stdout:331: left oid 29 (ObjNum 147 snap 29 seq_num 147) 2024-09-17T17:30:40.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:30:40.394 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.394 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.403 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:40.435 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:40.433+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:40.492 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 2] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:30:40.492 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:40.492 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T17:30:40.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-17T17:30:40.503 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:40.501+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:40.506 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.506 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.514 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:40.605 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:40.602+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:40.618 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.618 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.627 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:40.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:40.706 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:40.704+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:40.730 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.730 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.739 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:40.786 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:40.786 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout:moved 0 / 200 (0%) 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout:avg 12.5 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 2.69258 -> 2.69258 (expected baseline 3.42327) 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.3 with 8 -> 8 pgs (0.64 -> 0.64 * mean) 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.1 with 19 -> 19 pgs (1.52 -> 1.52 * mean) 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:40.809 INFO:teuthology.orchestra.run.smithi007.stdout:oload 1000 2024-09-17T17:30:40.810 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 3 2024-09-17T17:30:40.810 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T17:30:40.810 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0005 2024-09-17T17:30:40.810 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0049 2024-09-17T17:30:40.810 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:40.804+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T17:30:40.810 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:40.822 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T17:30:40.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-17T17:30:40.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:40.843 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:40.844 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:40.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:40.854 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:40.854 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:40.875 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:40.897 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:40.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:40.937 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:40.934+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:40.942 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:40.963 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:40.984 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:40.985 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:40.998 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:311 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 431326~1305378,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.025640396999997, 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085766909999999991, 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:40.999 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 311 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.000 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970414+0000", 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.8299999999999997e-07 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970418+0000", 2024-09-17T17:30:41.001 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9500000000000003e-06 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970425+0000", 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.452e-06 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970429+0000", 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4240000000000002e-06 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.002 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.971012+0000", 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00058375099999999997 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.971104+0000", 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1195999999999999e-05 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978990+0000", 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078864350000000007 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.003 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:312 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 0~1736704,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.025611662000003, 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50817852500000005, 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.004 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 312 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:41.005 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1600000000000002e-07 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970445+0000", 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7999999999999999e-06 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.006 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970451+0000", 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0970000000000002e-06 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970456+0000", 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6009999999999997e-06 2024-09-17T17:30:41.007 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.008 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.008 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.008 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976257+0000", 2024-09-17T17:30:41.008 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005800889 2024-09-17T17:30:41.008 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.008 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.009 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.009 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976348+0000", 2024-09-17T17:30:41.009 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1614999999999996e-05 2024-09-17T17:30:41.009 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.009 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.009 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.010 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478621+0000", 2024-09-17T17:30:41.010 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50227210700000002 2024-09-17T17:30:41.010 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.010 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.010 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.010 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.011 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.011 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.ds0 43/32 ECSubReadReply(tid=23, attrs_read=0))", 2024-09-17T17:30:41.011 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:41.011 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.01900097, 2024-09-17T17:30:41.011 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.501606106, 2024-09-17T17:30:41.011 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.012 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.012 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.012 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.012 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.012 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.012 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:41.013 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.013 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.013 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.013 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.013 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:41.013 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.014 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.014 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.014 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.014 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977054+0000", 2024-09-17T17:30:41.015 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.16e-07 2024-09-17T17:30:41.015 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.015 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.015 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.015 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978014+0000", 2024-09-17T17:30:41.016 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096069199999999997 2024-09-17T17:30:41.016 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.016 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.016 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.016 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477737+0000", 2024-09-17T17:30:41.016 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.49972330100000001 2024-09-17T17:30:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477745+0000", 2024-09-17T17:30:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2130000000000002e-06 2024-09-17T17:30:41.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.018 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.018 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.018 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477802+0000", 2024-09-17T17:30:41.018 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7555e-05 2024-09-17T17:30:41.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.019 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.019 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.019 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478659+0000", 2024-09-17T17:30:41.019 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00085652899999999997 2024-09-17T17:30:41.019 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.019 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.020 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.020 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.020 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.020 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=31, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:30:41.020 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:41.020 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 40.638182837999999, 2024-09-17T17:30:41.021 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53920807500000001, 2024-09-17T17:30:41.021 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.021 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.021 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.021 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.021 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.022 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.022 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:41.022 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.022 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.022 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.022 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.023 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:41.023 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.023 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.023 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.023 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.024 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:41.024 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:30:41.024 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.024 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.024 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.024 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357876+0000", 2024-09-17T17:30:41.025 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.092e-06 2024-09-17T17:30:41.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.025 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.025 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.025 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896536+0000", 2024-09-17T17:30:41.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53865993499999998 2024-09-17T17:30:41.026 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.026 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.026 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.026 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896547+0000", 2024-09-17T17:30:41.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1608e-05 2024-09-17T17:30:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896653+0000", 2024-09-17T17:30:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000105807 2024-09-17T17:30:41.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.028 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.028 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.028 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897079+0000", 2024-09-17T17:30:41.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042597099999999998 2024-09-17T17:30:41.028 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.029 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.029 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.029 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.029 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.029 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=32, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:30:41.030 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:41.030 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 40.098551669000003, 2024-09-17T17:30:41.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222760399, 2024-09-17T17:30:41.030 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.030 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.030 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.031 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.031 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.031 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.031 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:41.031 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.031 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.032 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.032 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.032 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:41.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.033 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.033 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.033 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897503+0000", 2024-09-17T17:30:41.033 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.68e-07 2024-09-17T17:30:41.033 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.033 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.034 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.034 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897507+0000", 2024-09-17T17:30:41.034 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7709999999999997e-06 2024-09-17T17:30:41.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.034 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120082+0000", 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222574243 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120095+0000", 2024-09-17T17:30:41.035 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.291e-05 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120199+0000", 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010402499999999999 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120262+0000", 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3781999999999993e-05 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.037 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 51/32 ECSubWrite(tid=61, reqid=osd.3.0:61, at_version=51'39, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.655932311999997, 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007939804, 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.038 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.039 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340122+0000", 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4200000000000005e-07 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.040 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340146+0000", 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3578e-05 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340199+0000", 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3127999999999997e-05 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.041 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340208+0000", 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3589999999999997e-06 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340279+0000", 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0772000000000006e-05 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.042 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340289+0000", 2024-09-17T17:30:41.043 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9860000000000006e-06 2024-09-17T17:30:41.043 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.043 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.043 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.043 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347977+0000", 2024-09-17T17:30:41.043 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076882779999999998 2024-09-17T17:30:41.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.044 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.044 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348061+0000", 2024-09-17T17:30:41.044 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3861000000000002e-05 2024-09-17T17:30:41.044 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.045 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.045 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.045 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.045 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s1 51/35 ECSubWrite(tid=41, reqid=osd.10.0:41, at_version=51'30, trim_to=0'0, roll_forward_to=49'28))", 2024-09-17T17:30:41.045 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:41.046 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.655729972000003, 2024-09-17T17:30:41.046 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0077529670000000004, 2024-09-17T17:30:41.046 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.046 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.046 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.046 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.047 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340324+0000", 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.049 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.049 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340350+0000", 2024-09-17T17:30:41.049 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5931e-05 2024-09-17T17:30:41.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340417+0000", 2024-09-17T17:30:41.050 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6718999999999996e-05 2024-09-17T17:30:41.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.050 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340426+0000", 2024-09-17T17:30:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0489999999999993e-06 2024-09-17T17:30:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340489+0000", 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3219000000000006e-05 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340498+0000", 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6740000000000003e-06 2024-09-17T17:30:41.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348033+0000", 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075347440000000003 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348077+0000", 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4076000000000003e-05 2024-09-17T17:30:41.053 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s5 51/35 ECSubWrite(tid=122, reqid=osd.13.0:122, at_version=51'53, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.343918+0000", 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.652135901000001, 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079303870000000005, 2024-09-17T17:30:41.054 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343918+0000", 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343918+0000", 2024-09-17T17:30:41.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343919+0000", 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2079999999999999e-06 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343944+0000", 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4630999999999998e-05 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.056 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343977+0000", 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3506000000000001e-05 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343985+0000", 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2309999999999996e-06 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.346574+0000", 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0025890850000000001 2024-09-17T17:30:41.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.346583+0000", 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7260000000000008e-06 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351805+0000", 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0052215819999999998 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351848+0000", 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3417999999999997e-05 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:412 3.ds0 3:badfad2a:::smithi10238019-37:head [append 0~655360] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.639336858999997, 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097729180000000002, 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.059 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 412 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356719+0000", 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.255e-06 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357255+0000", 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00053548099999999998 2024-09-17T17:30:41.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357270+0000", 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5045000000000001e-05 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357278+0000", 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8429999999999999e-06 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357346+0000", 2024-09-17T17:30:41.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7186000000000004e-05 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357460+0000", 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114388 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.358484+0000", 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023641 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366119+0000", 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076354270000000002 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366416+0000", 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00029701100000000002 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366490+0000", 2024-09-17T17:30:41.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3640999999999995e-05 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:415 3.ds0 3:badfad2a:::smithi10238019-37:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.634143197999997, 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089535759999999995, 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.065 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 415 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361911+0000", 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7e-07 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361913+0000", 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2350000000000002e-06 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361919+0000", 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4750000000000001e-06 2024-09-17T17:30:41.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361922+0000", 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3110000000000001e-06 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366710+0000", 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047879840000000003 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366770+0000", 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9886000000000002e-05 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367045+0000", 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027536599999999998 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370540+0000", 2024-09-17T17:30:41.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034951610000000001 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370583+0000", 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2682999999999998e-05 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370864+0000", 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00028100500000000002 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.071 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:416 3.ds0 3:badfad2a:::smithi10238019-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.634126172000002, 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096567449999999996, 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.072 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 416 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361928+0000", 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5400000000000002e-07 2024-09-17T17:30:41.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361929+0000", 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.471e-06 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361934+0000", 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1100000000000002e-06 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361937+0000", 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1499999999999999e-06 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367591+0000", 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005653534 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:41.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367664+0000", 2024-09-17T17:30:41.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2779000000000002e-05 2024-09-17T17:30:41.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370895+0000", 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0032314409999999998 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370943+0000", 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7886999999999997e-05 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.371584+0000", 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000641119 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 56/32 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=56'57, trim_to=0'0, roll_forward_to=53'55))", 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:41.079 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 33.652434, 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089122520000000007, 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.92e-07 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343638+0000", 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8074e-05 2024-09-17T17:30:41.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343672+0000", 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343680+0000", 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8979999999999996e-06 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343804+0000", 2024-09-17T17:30:41.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012350499999999999 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343814+0000", 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0353000000000001e-05 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352491+0000", 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086771880000000006 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352532+0000", 2024-09-17T17:30:41.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0837999999999998e-05 2024-09-17T17:30:41.084 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.084 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.084 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.085 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.16s5 68/67 ECSubWrite(tid=198, reqid=osd.13.0:198, at_version=68'119, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:41.085 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.463558+0000", 2024-09-17T17:30:41.085 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.532495439000002, 2024-09-17T17:30:41.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0077676009999999998, 2024-09-17T17:30:41.086 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.086 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.086 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.086 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463558+0000", 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463558+0000", 2024-09-17T17:30:41.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463559+0000", 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1900000000000001e-07 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463583+0000", 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3561e-05 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463618+0000", 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5172000000000001e-05 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463625+0000", 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1149999999999996e-06 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463687+0000", 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2220999999999999e-05 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.463708+0000", 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1178999999999998e-05 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.471286+0000", 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075776359999999996 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.471326+0000", 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9898000000000003e-05 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 74/32 ECSubWrite(tid=148, reqid=client.4566.0:736, at_version=74'114, trim_to=0'0, roll_forward_to=74'111))", 2024-09-17T17:30:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 15.610338128, 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94858601300000001, 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5599999999999995e-07 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385733+0000", 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7152000000000001e-05 2024-09-17T17:30:41.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333728+0000", 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94799460099999999 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333734+0000", 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6109999999999999e-06 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.094 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333806+0000", 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1377000000000005e-05 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333813+0000", 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2969999999999998e-06 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334272+0000", 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000458956 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334302+0000", 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9462999999999999e-05 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.16s5 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 14.63220623, 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011160745999999999, 2024-09-17T17:30:41.096 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363848+0000", 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7400000000000003e-07 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363866+0000", 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8411999999999999e-05 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363893+0000", 2024-09-17T17:30:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6346000000000002e-05 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363899+0000", 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5220000000000004e-06 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363943+0000", 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3933000000000001e-05 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363948+0000", 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0810000000000001e-06 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.374983+0000", 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011034284 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.375008+0000", 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5593999999999999e-05 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.100 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 78/32 ECSubWriteReply(tid=78, last_complete=78'67, committed=1, applied=1))", 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.351804108, 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.370820394, 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4000000000000001e-07 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644254+0000", 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.924e-06 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014178+0000", 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.369923422 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014192+0000", 2024-09-17T17:30:41.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4722999999999999e-05 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014343+0000", 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015040400000000001 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.015070+0000", 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072728099999999998 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.104 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 78/32 ECSubWriteReply(tid=79, last_complete=78'67, committed=1, applied=1))", 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.979958684, 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.47947288999999998, 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016096+0000", 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7000000000000005e-07 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016099+0000", 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2660000000000002e-06 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495222+0000", 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.47912339300000001 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495235+0000", 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3134e-05 2024-09-17T17:30:41.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495445+0000", 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000209302 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495568+0000", 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000123225 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 85/84 [PushOp(3:497089da:::smithi10238019-9:13, version: 71'47, data_included: [0~36864], data_size: 36864, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:13@71'47, size: 147456, copy_subset: [], clone_subset: {}, snapset: 1b={13: [13, 10], 1b: [1b, 1a, 18, 17, 16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 147456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 4.5559622429999997, 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.43750746400000001, 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:30:41.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440092+0000", 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8999999999999996e-07 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440171+0000", 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9203999999999996e-05 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877162+0000", 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.436990934 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877172+0000", 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0164e-05 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877272+0000", 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9265000000000006e-05 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877599+0000", 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00032730699999999999 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 85/84 [PushOp(3:497089da:::smithi10238019-9:1b, version: 82'53, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:1b@82'53, size: 819200, copy_subset: [], clone_subset: {}, snapset: 1b={13: [13, 10], 1b: [1b, 1a, 18, 17, 16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 819200, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 4.1063362940000001, 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.46291758199999999, 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889718+0000", 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8200000000000008e-07 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889996+0000", 2024-09-17T17:30:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027726200000000002 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351714+0000", 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.46171880599999998 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351746+0000", 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1831000000000001e-05 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351912+0000", 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016581899999999999 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.352635+0000", 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000722882 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 86/84 [PushOp(3:497089da:::smithi10238019-9:head, version: 82'58, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:head@82'58, size: 3112960, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 3.6442327629999998, 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.129055472, 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8800000000000002e-07 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.352494+0000", 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067254700000000001 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480390+0000", 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.12789568900000001 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480399+0000", 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7610000000000002e-06 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480484+0000", 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4571000000000001e-05 2024-09-17T17:30:41.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480876+0000", 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039231599999999998 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.120 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:41.122 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:41.122 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:41.119+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:moved 0 / 200 (0%) 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:avg 12.5 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 2.69258 -> 2.69258 (expected baseline 3.42327) 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.3 with 8 -> 8 pgs (0.64 -> 0.64 * mean) 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.1 with 19 -> 19 pgs (1.52 -> 1.52 * mean) 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:oload 1000 2024-09-17T17:30:41.134 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 0.05 2024-09-17T17:30:41.134 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T17:30:41.134 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0005 2024-09-17T17:30:41.134 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0049 2024-09-17T17:30:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T17:30:41.146 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T17:30:41.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T17:30:41.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:41.222 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:41.220+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:41.229 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:41.249 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:41.266 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:41.266 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:41.267 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:41.267 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:41.271 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:41.270+0000 7f29177fe640 0 -- 172.21.15.7:0/2798910463 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f291821f390 msgr2=0x7f2918221780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:41.274 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:41.323 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:41.321+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:41.341 INFO:tasks.rados.rados.0.smithi102.stdout:331: done (1 left) 2024-09-17T17:30:41.341 INFO:tasks.rados.rados.0.smithi102.stdout:344: done (0 left) 2024-09-17T17:30:41.341 INFO:tasks.rados.rados.0.smithi102.stdout:345: rollback oid 15 current snap is 30 2024-09-17T17:30:41.341 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 15 to 29 2024-09-17T17:30:41.386 INFO:tasks.rados.rados.0.smithi102.stdout:346: setattr oid 21 current snap is 30 2024-09-17T17:30:41.387 INFO:tasks.rados.rados.0.smithi102.stdout:347: snap_remove snap 26 2024-09-17T17:30:41.387 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:41.387 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:41.387 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:41.388 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:41.388 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:30:41.390 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:41.431 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:41.428+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:41.494 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:41.494 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:41.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:41.494 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:41.494 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.494 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=24, last_complete=36'10, committed=1, applied=1))", 2024-09-17T17:30:41.496 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:41.496 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 53.036583612999998, 2024-09-17T17:30:41.496 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53983539199999997, 2024-09-17T17:30:41.496 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:41.498 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454674+0000", 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0790000000000002e-06 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992637+0000", 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53796236600000003 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992644+0000", 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5820000000000004e-06 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994469+0000", 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018243350000000001 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994505+0000", 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6300999999999997e-05 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:173 3.cs0 3:30756a83:::smithi10238019-35:2 [read 297244~276196,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:41.501 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 52.501160978999998, 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011614354, 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.502 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 173 2024-09-17T17:30:41.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.503 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.503 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.503 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.503 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:41.503 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.504 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990092+0000", 2024-09-17T17:30:41.504 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.505 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.505 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.505 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990093+0000", 2024-09-17T17:30:41.505 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:30:41.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990099+0000", 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2090000000000002e-06 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990111+0000", 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1372000000000001e-05 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990118+0000", 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.126e-06 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.507 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990197+0000", 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8665999999999995e-05 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.508 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990299+0000", 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000102397 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001707+0000", 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011407690999999999 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:174 3.cs0 3:30756a83:::smithi10238019-35:2 [read 501714~71726,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:41.509 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 52.501115519999999, 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011875784, 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 174 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9400000000000005e-07 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990142+0000", 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.557e-06 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.512 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990195+0000", 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3012e-05 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990206+0000", 2024-09-17T17:30:41.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0876000000000001e-05 2024-09-17T17:30:41.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.514 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.514 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.514 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990783+0000", 2024-09-17T17:30:41.514 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00057770200000000003 2024-09-17T17:30:41.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.515 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990850+0000", 2024-09-17T17:30:41.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6168999999999996e-05 2024-09-17T17:30:41.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.516 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.516 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.002014+0000", 2024-09-17T17:30:41.516 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011163874000000001 2024-09-17T17:30:41.516 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.516 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.517 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.517 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.517 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.517 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:175 3.cs0 3:30756a83:::smithi10238019-35:2 [read 194830~378610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:41.517 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 52.501028961999999, 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.30022850299999998, 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 175 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.518 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990225+0000", 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8199999999999998e-07 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990230+0000", 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7809999999999996e-06 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990371+0000", 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014071299999999999 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990381+0000", 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0061000000000001e-05 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994702+0000", 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043214899999999999 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994769+0000", 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6600999999999995e-05 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290453+0000", 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.29568397499999999 2024-09-17T17:30:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=28, attrs_read=0))", 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 52.490055945999998, 2024-09-17T17:30:41.523 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.289267987, 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.524 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001198+0000", 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6000000000000004e-07 2024-09-17T17:30:41.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001279+0000", 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1168000000000006e-05 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289843+0000", 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.28856357599999999 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.526 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289854+0000", 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1749999999999999e-05 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289975+0000", 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012060100000000001 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290465+0000", 2024-09-17T17:30:41.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049033200000000001 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:207 3.cs0 3:30d62bba:::smithi10238019-34:head [copy-from ver 2] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 50.785016880999997, 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011310328, 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.528 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 207 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.529 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706236+0000", 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706238+0000", 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9230000000000001e-06 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706245+0000", 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5089999999999999e-06 2024-09-17T17:30:41.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706261+0000", 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6487999999999999e-05 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706268+0000", 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2859999999999996e-06 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.531 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.532 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706339+0000", 2024-09-17T17:30:41.532 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1600999999999994e-05 2024-09-17T17:30:41.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.532 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.532 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.532 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.706451+0000", 2024-09-17T17:30:41.533 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000112071 2024-09-17T17:30:41.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.533 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.533 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.533 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.714618+0000", 2024-09-17T17:30:41.533 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0081665550000000007 2024-09-17T17:30:41.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.534 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.534 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.534 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.715772+0000", 2024-09-17T17:30:41.534 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011545959999999999 2024-09-17T17:30:41.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.717464+0000", 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001691641 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.717547+0000", 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2657999999999994e-05 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.1s1 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.793364629000003, 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011087913, 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.537 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.538 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697889+0000", 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.55e-07 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.539 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697911+0000", 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1744999999999999e-05 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.697975+0000", 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4414999999999999e-05 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.540 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698108+0000", 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013266499999999999 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698174+0000", 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6011000000000002e-05 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.698183+0000", 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2539999999999998e-06 2024-09-17T17:30:41.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.542 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.542 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.542 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.708938+0000", 2024-09-17T17:30:41.542 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010755602 2024-09-17T17:30:41.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.542 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.543 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.543 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.708977+0000", 2024-09-17T17:30:41.543 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8065999999999998e-05 2024-09-17T17:30:41.543 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.543 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.543 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.544 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.544 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:316 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (56) in=7b] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:41.544 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:41.544 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.501795180999999, 2024-09-17T17:30:41.545 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041035350999999998, 2024-09-17T17:30:41.545 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.545 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.545 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.545 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.545 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.546 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.546 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 316 2024-09-17T17:30:41.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.546 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.546 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.547 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.547 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:41.547 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.547 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.547 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.548 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:41.548 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.548 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.548 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.548 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.548 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989459+0000", 2024-09-17T17:30:41.549 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2200000000000004e-07 2024-09-17T17:30:41.549 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.549 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.549 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.549 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989462+0000", 2024-09-17T17:30:41.550 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.287e-06 2024-09-17T17:30:41.550 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.550 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.550 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.550 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989469+0000", 2024-09-17T17:30:41.550 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7229999999999998e-06 2024-09-17T17:30:41.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.551 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.551 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.551 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989472+0000", 2024-09-17T17:30:41.551 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.428e-06 2024-09-17T17:30:41.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.552 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.552 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.552 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990430+0000", 2024-09-17T17:30:41.552 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00095784099999999997 2024-09-17T17:30:41.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.553 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.553 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.553 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990525+0000", 2024-09-17T17:30:41.553 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5129000000000003e-05 2024-09-17T17:30:41.553 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.553 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.554 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.554 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990951+0000", 2024-09-17T17:30:41.554 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042552399999999999 2024-09-17T17:30:41.554 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.554 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.554 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991309+0000", 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000358632 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030441+0000", 2024-09-17T17:30:41.555 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039131753999999998 2024-09-17T17:30:41.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.556 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.556 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.556 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030493+0000", 2024-09-17T17:30:41.556 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2510999999999999e-05 2024-09-17T17:30:41.556 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.557 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.557 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.557 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.557 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:317 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:41.557 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:41.558 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.501761844999997, 2024-09-17T17:30:41.558 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041694146000000001, 2024-09-17T17:30:41.558 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.558 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.558 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.558 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.559 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.559 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.559 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 317 2024-09-17T17:30:41.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.559 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.559 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.560 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.560 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:41.560 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.560 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.560 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.561 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:41.561 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.561 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.561 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.561 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989492+0000", 2024-09-17T17:30:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3099999999999998e-07 2024-09-17T17:30:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989494+0000", 2024-09-17T17:30:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3329999999999999e-06 2024-09-17T17:30:41.563 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.563 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.563 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.563 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989501+0000", 2024-09-17T17:30:41.563 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.516e-06 2024-09-17T17:30:41.563 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.564 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.564 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.564 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989505+0000", 2024-09-17T17:30:41.564 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.912e-06 2024-09-17T17:30:41.564 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.565 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.565 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.565 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991167+0000", 2024-09-17T17:30:41.565 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001662188 2024-09-17T17:30:41.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.565 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.566 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:41.566 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991214+0000", 2024-09-17T17:30:41.566 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6944000000000003e-05 2024-09-17T17:30:41.566 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.566 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.566 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.567 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030733+0000", 2024-09-17T17:30:41.567 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039519406999999999 2024-09-17T17:30:41.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.567 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.567 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.567 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030782+0000", 2024-09-17T17:30:41.568 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9338000000000001e-05 2024-09-17T17:30:41.568 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.568 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.568 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.568 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.031186+0000", 2024-09-17T17:30:41.568 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000403077 2024-09-17T17:30:41.569 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.569 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.569 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.569 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.569 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:429 3.cs0 3:30d62bba:::smithi10238019-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:41.570 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:41.570 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.098350314999998, 2024-09-17T17:30:41.570 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014137011, 2024-09-17T17:30:41.570 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.570 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.571 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.571 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.571 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.571 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.571 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 429 2024-09-17T17:30:41.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.572 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.572 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.572 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.572 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:41.572 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:41.573 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3299999999999998e-07 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392905+0000", 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6869999999999999e-06 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392912+0000", 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6250000000000001e-06 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.574 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392916+0000", 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7330000000000003e-06 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397110+0000", 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041939450000000001 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397209+0000", 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8961000000000003e-05 2024-09-17T17:30:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402765+0000", 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055556420000000004 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402840+0000", 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4943000000000004e-05 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.407040+0000", 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042001419999999996 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:436 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 12] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.098253417000002, 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012854290000000001, 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.577 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 436 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.578 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3700000000000001e-07 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393005+0000", 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4190000000000004e-06 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393008+0000", 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7009999999999998e-06 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393023+0000", 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4545999999999999e-05 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397903+0000", 2024-09-17T17:30:41.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048805719999999997 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397964+0000", 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0520000000000003e-05 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.401036+0000", 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030720470000000001 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402722+0000", 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001685513 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405761+0000", 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030398790000000001 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405854+0000", 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2756000000000004e-05 2024-09-17T17:30:41.582 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:486 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 34.112246923999997, 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75510421100000003, 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.583 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 486 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:41.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379007+0000", 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379011+0000", 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0720000000000001e-06 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379019+0000", 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9580000000000007e-06 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379025+0000", 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6799999999999998e-06 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379992+0000", 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096721299999999997 2024-09-17T17:30:41.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380044+0000", 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2091e-05 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380358+0000", 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000313543 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.587 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380731+0000", 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00037292 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134034+0000", 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75330350700000004 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134111+0000", 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6497999999999998e-05 2024-09-17T17:30:41.588 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:487 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 34.112217782999998, 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.756014517, 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.589 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 487 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379036+0000", 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5499999999999998e-07 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379038+0000", 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.591 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379045+0000", 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8770000000000004e-06 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379050+0000", 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2299999999999999e-06 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.592 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380522+0000", 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001471909 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380575+0000", 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2821999999999997e-05 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134417+0000", 2024-09-17T17:30:41.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75384171300000002 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134482+0000", 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5789000000000003e-05 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.135050+0000", 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00056751999999999996 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.594 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:626 3.cs0 3:30756a83:::smithi10238019-35:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.992955945999999, 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021079339999999998, 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.595 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 626 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498298+0000", 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498301+0000", 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7530000000000002e-06 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498307+0000", 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9869999999999996e-06 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498310+0000", 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1259999999999998e-06 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499848+0000", 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001537912 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.598 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499903+0000", 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5763000000000001e-05 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500222+0000", 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00031825499999999998 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500848+0000", 2024-09-17T17:30:41.599 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062612400000000004 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519301+0000", 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018453431999999999 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519377+0000", 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5405000000000001e-05 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.600 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:627 3.cs0 3:30756a83:::smithi10238019-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.992926785999998, 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021540693, 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 627 2024-09-17T17:30:41.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498327+0000", 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.51e-07 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498329+0000", 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3989999999999998e-06 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498337+0000", 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5379999999999997e-06 2024-09-17T17:30:41.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498341+0000", 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4349999999999999e-06 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500569+0000", 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002227934 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:41.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500629+0000", 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9721e-05 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519409+0000", 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018780192000000001 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519465+0000", 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6062e-05 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519867+0000", 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000401961 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s6 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.990640481, 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26792300099999999, 2024-09-17T17:30:41.606 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:41.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500614+0000", 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6700000000000002e-07 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500747+0000", 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013382499999999999 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.608 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767249+0000", 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26650138800000001 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767261+0000", 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1785e-05 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767372+0000", 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111743 2024-09-17T17:30:41.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767387+0000", 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4548e-05 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768494+0000", 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011074990000000001 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768536+0000", 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1346e-05 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:639 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.967458475000001, 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67002614400000005, 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.611 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 639 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0500000000000003e-07 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1729999999999999e-06 2024-09-17T17:30:41.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523810+0000", 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1627000000000001e-05 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523815+0000", 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0420000000000002e-06 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525161+0000", 2024-09-17T17:30:41.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0013460169999999999 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525253+0000", 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1793999999999996e-05 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526004+0000", 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00075112400000000004 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527034+0000", 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001029713 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193749+0000", 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.66671533999999999 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193821+0000", 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1709000000000001e-05 2024-09-17T17:30:41.616 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:640 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.967426859, 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67078859899999999, 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.617 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 640 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523827+0000", 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.03e-07 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523829+0000", 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1739999999999999e-06 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523836+0000", 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3869999999999997e-06 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523841+0000", 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4089999999999996e-06 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527242+0000", 2024-09-17T17:30:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003401383 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527298+0000", 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5705000000000003e-05 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194159+0000", 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.666861484 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194218+0000", 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9039000000000002e-05 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194615+0000", 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039661500000000001 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.622 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 68/32 ECSubWrite(tid=118, reqid=client.4566.0:641, at_version=68'91, trim_to=0'0, roll_forward_to=66'89))", 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.966585743, 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.164368816, 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524694+0000", 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6256000000000001e-05 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687716+0000", 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.163021575 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687725+0000", 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9390000000000003e-06 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687798+0000", 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900999999999999e-05 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687809+0000", 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.153e-05 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689007+0000", 2024-09-17T17:30:41.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001197844 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689036+0000", 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9342000000000001e-05 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 78/32 ECSubWrite(tid=154, reqid=osd.3.0:154, at_version=78'120, trim_to=0'0, roll_forward_to=78'118))", 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:30:41.627 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.234868392999999, 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.088487576999999998, 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256386+0000", 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3900000000000004e-07 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256409+0000", 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3475000000000001e-05 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342362+0000", 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.085952671999999994 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342372+0000", 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0285000000000001e-05 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343593+0000", 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001220858 2024-09-17T17:30:41.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343604+0000", 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1078e-05 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344835+0000", 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0012308309999999999 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344872+0000", 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7539000000000001e-05 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.632 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:41.633 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:41.634 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:41.634 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:41.636 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:41.637 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:41.637 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:41.631+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:41.734 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:41.733+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:41.740 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:41.740 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:41.740 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:41.740 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:41.749 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:41.835 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:41.833+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:41.853 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:41.853 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:41.862 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:41.936 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:41.934+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:41.965 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:41.965 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 37/35 ECSubWriteReply(tid=17, last_complete=37'13, committed=1, applied=1))", 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 52.962012031999997, 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.068701408000000005, 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000937+0000", 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6449999999999998e-06 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068896+0000", 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.067959031000000003 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068909+0000", 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3416e-05 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069008+0000", 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8690999999999994e-05 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069634+0000", 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062589099999999995 2024-09-17T17:30:41.969 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 51.279080774000001, 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079875469999999994, 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.970 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2e-07 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684336+0000", 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047134800000000001 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684439+0000", 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010291800000000001 2024-09-17T17:30:41.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684449+0000", 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.027e-05 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684596+0000", 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000146749 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684607+0000", 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1421e-05 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691787+0000", 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071804069999999998 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691851+0000", 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3913999999999998e-05 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 51.277590367999998, 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089994210000000005, 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4100000000000001e-07 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.976 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685373+0000", 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9063999999999999e-05 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687383+0000", 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002009355 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687391+0000", 2024-09-17T17:30:41.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9699999999999999e-06 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687457+0000", 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5834999999999996e-05 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9990000000000003e-06 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694321+0000", 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068546010000000001 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694354+0000", 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2456000000000002e-05 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.979 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:191 3.5s0 3:a4d61510:::smithi10238019-32:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 51.275507505999997, 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.008961732, 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 191 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.980 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687439+0000", 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1799999999999999e-06 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687504+0000", 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4770000000000005e-05 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687516+0000", 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2007e-05 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687531+0000", 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4803e-05 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687595+0000", 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3697999999999999e-05 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687756+0000", 2024-09-17T17:30:41.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000161846 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.690070+0000", 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0023137639999999998 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695782+0000", 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057116959999999996 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.984 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695908+0000", 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000126339 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.696399+0000", 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049062900000000002 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.985 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:407 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.624867356999999, 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011701008000000001, 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 407 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.986 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338080+0000", 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2819999999999999e-06 2024-09-17T17:30:41.987 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6289999999999999e-06 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338108+0000", 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9514e-05 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.988 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338117+0000", 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.132e-06 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1446999999999995e-05 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338196+0000", 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6350000000000006e-06 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.989 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341655+0000", 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034593860000000001 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342199+0000", 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054402400000000005 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349778+0000", 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075789589999999997 2024-09-17T17:30:41.990 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:408 3.5s0 3:a96b502c:::smithi10238019-42:head [read 53220~651292,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.624705429999999, 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012064669, 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.991 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 408 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:41.992 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.993 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.993 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.993 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.993 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338240+0000", 2024-09-17T17:30:41.993 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338257+0000", 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6643e-05 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338270+0000", 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3081000000000001e-05 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.994 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338277+0000", 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3780000000000003e-06 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338331+0000", 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4286999999999997e-05 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338338+0000", 2024-09-17T17:30:41.995 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3609999999999997e-06 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342681+0000", 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043432749999999997 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342775+0000", 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4392e-05 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.996 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.350304+0000", 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007528435 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:409 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.624592675000002, 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012665852, 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 409 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.998 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338353+0000", 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.09e-07 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338358+0000", 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7859999999999996e-06 2024-09-17T17:30:41.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338366+0000", 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6049999999999998e-06 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338372+0000", 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1739999999999997e-06 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.000 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338439+0000", 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7039999999999995e-05 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338446+0000", 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1160000000000001e-06 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343114+0000", 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0046672179999999999 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.001 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343207+0000", 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3170000000000004e-05 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351018+0000", 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078109340000000003 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.002 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 51/35 ECSubWrite(tid=122, reqid=osd.13.0:122, at_version=51'53, trim_to=0'0, roll_forward_to=49'49))", 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.618908867000002, 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073809449999999999, 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.003 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344036+0000", 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1399999999999996e-07 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344057+0000", 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0273000000000001e-05 2024-09-17T17:30:42.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344096+0000", 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9045000000000003e-05 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344104+0000", 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2050000000000002e-06 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.005 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344311+0000", 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00020691500000000001 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.344321+0000", 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0307e-05 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351381+0000", 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0070600910000000001 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.006 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351417+0000", 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5395000000000002e-05 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.610438295000002, 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.988933536, 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.007 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.008 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352507+0000", 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1500000000000003e-07 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352532+0000", 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4485999999999999e-05 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.009 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340513+0000", 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.98798163100000003 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340522+0000", 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9379999999999998e-06 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340625+0000", 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010273 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.010 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340633+0000", 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341398+0000", 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00076522600000000004 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341440+0000", 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1548999999999998e-05 2024-09-17T17:30:42.011 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/35 ECSubWrite(tid=50, reqid=client.4566.0:441, at_version=0'0, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.099380775, 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.121580543, 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.012 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.013 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863565+0000", 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6729999999999999e-06 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863588+0000", 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2756e-05 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984520+0000", 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.120931489 2024-09-17T17:30:42.014 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984547+0000", 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6877999999999999e-05 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984631+0000", 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4246000000000006e-05 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.015 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984638+0000", 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3509999999999996e-06 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985117+0000", 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047851700000000003 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985144+0000", 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7633000000000001e-05 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.016 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 53/32 ECSubWrite(tid=72, reqid=osd.3.0:72, at_version=53'53, trim_to=0'0, roll_forward_to=52'49))", 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 37.600396459999999, 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0070743660000000003, 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.017 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362548+0000", 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362549+0000", 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5400000000000001e-07 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.018 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362568+0000", 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9213000000000001e-05 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362606+0000", 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7647000000000001e-05 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.362614+0000", 2024-09-17T17:30:42.019 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2579999999999995e-06 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363252+0000", 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00063853300000000005 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.363262+0000", 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9359999999999994e-06 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.020 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369572+0000", 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063095340000000003 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:04.369622+0000", 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0590999999999997e-05 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:42.021 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.462290242000002, 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.090576745, 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.022 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500656+0000", 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0430000000000002e-06 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500772+0000", 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000115338 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590079+0000", 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089307485000000006 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590103+0000", 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4131e-05 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590159+0000", 2024-09-17T17:30:42.024 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5417000000000002e-05 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590177+0000", 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7819e-05 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591201+0000", 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023939 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.025 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591231+0000", 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0573e-05 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 75/74 ECSubWrite(tid=94, reqid=osd.10.0:94, at_version=75'58, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 15.610822616, 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086672569999999994, 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.026 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.027 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352123+0000", 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352143+0000", 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0440000000000001e-05 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352170+0000", 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6503e-05 2024-09-17T17:30:42.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352177+0000", 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6429999999999995e-06 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352236+0000", 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8711000000000001e-05 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.029 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352245+0000", 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360755+0000", 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085095469999999993 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360789+0000", 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4678999999999998e-05 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.030 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 78/74 ECSubWrite(tid=102, reqid=client.4566.0:771, at_version=78'64, trim_to=0'0, roll_forward_to=78'60))", 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.581091275, 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96796263999999999, 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.031 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381854+0000", 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4e-07 2024-09-17T17:30:42.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381870+0000", 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6076e-05 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348683+0000", 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96681319499999996 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.033 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348717+0000", 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348821+0000", 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010401800000000001 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348833+0000", 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2172e-05 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.034 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349780+0000", 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00094660899999999995 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349816+0000", 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6025999999999997e-05 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.035 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 79/74 ECSubWrite(tid=107, reqid=client.4566.0:787, at_version=79'59, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.363281+0000", 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.599663637999999, 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071523009999999998, 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.363281+0000", 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.036 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.363281+0000", 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.363282+0000", 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2279999999999999e-06 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.363341+0000", 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9138000000000002e-05 2024-09-17T17:30:42.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365598+0000", 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00225684 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365604+0000", 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1419999999999998e-06 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.038 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365663+0000", 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8513e-05 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365673+0000", 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.008e-05 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.370382+0000", 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047094069999999997 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.370433+0000", 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0952999999999998e-05 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 79/74 ECSubWrite(tid=108, reqid=client.4566.0:788, at_version=79'60, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.597385223, 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010065843999999999, 2024-09-17T17:30:42.040 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365560+0000", 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5199999999999995e-07 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.043 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365578+0000", 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8145999999999999e-05 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374667+0000", 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.009088245 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374697+0000", 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0402000000000001e-05 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374763+0000", 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5912000000000002e-05 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374776+0000", 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2497000000000001e-05 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375591+0000", 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00081505399999999995 2024-09-17T17:30:42.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375625+0000", 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4635999999999998e-05 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 80/74 ECSubWrite(tid=112, reqid=osd.10.0:111, at_version=0'0, trim_to=0'0, roll_forward_to=80'62))", 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.610561391999999, 2024-09-17T17:30:42.046 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076437279999999998, 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352384+0000", 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0249999999999999e-06 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352400+0000", 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5628999999999999e-05 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352437+0000", 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6813000000000001e-05 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352448+0000", 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0882e-05 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352672+0000", 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00022395999999999999 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.049 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352683+0000", 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1656e-05 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.359996+0000", 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073125120000000002 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.360027+0000", 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1251000000000003e-05 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.050 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.13.22:9 3.5s0 3:a4d61510:::smithi10238019-32:head [assert-version v27,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 5.6208539819999999, 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015638470000000002, 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.051 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342092+0000", 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7e-06 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342098+0000", 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8690000000000001e-06 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342114+0000", 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5954999999999999e-05 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342122+0000", 2024-09-17T17:30:42.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3919999999999997e-06 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342179+0000", 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7768e-05 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342187+0000", 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3819999999999998e-06 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348515+0000", 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063278780000000003 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348672+0000", 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015732800000000001 2024-09-17T17:30:42.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.357729+0000", 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090571980000000007 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 85/76 ECSubWrite(tid=194, reqid=osd.12.0:193, at_version=0'0, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 5.6106415460000001, 2024-09-17T17:30:42.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0091352759999999995, 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:30:42.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352304+0000", 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4e-07 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352316+0000", 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1864e-05 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352360+0000", 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4478999999999999e-05 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352370+0000", 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5829999999999996e-06 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352437+0000", 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7521000000000006e-05 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352447+0000", 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2180000000000005e-06 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361372+0000", 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089248839999999992 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361438+0000", 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6786999999999996e-05 2024-09-17T17:30:42.060 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 85/74 ECSubWrite(tid=125, reqid=osd.10.0:125, at_version=85'76, trim_to=0'0, roll_forward_to=83'74))", 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 5.6100072120000002, 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085212919999999998, 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:42.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352938+0000", 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.04e-07 2024-09-17T17:30:42.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352958+0000", 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0486e-05 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353044+0000", 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5852000000000004e-05 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353050+0000", 2024-09-17T17:30:42.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2280000000000003e-06 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353110+0000", 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9478999999999998e-05 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353121+0000", 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1426e-05 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361426+0000", 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0083047659999999999 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361459+0000", 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2851000000000001e-05 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:42.065 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:42.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:42.067 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:42.065+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:42.167 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:42.165+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:42.183 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:42.183 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:42.183 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:42.183 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:42.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:42.269 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:42.267+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:42.298 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:42.299 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:42.299 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:42.299 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:42.299 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:42.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:42.343 INFO:tasks.rados.rados.0.smithi102.stdout:347: done (2 left) 2024-09-17T17:30:42.343 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 36 current snap is 30 2024-09-17T17:30:42.343 INFO:tasks.rados.rados.0.smithi102.stdout:348: seq_num 152 ranges {2015232=147456} 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout:moved 9 / 200 (4.5%) 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout:avg 12.5 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 2.69258 -> 2.78388 (expected baseline 3.42327) 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.3 with 8 -> 9 pgs (0.64 -> 0.72 * mean) 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.1 with 19 -> 16 pgs (1.52 -> 1.28 * mean) 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:42.344 INFO:teuthology.orchestra.run.smithi007.stdout:oload 110 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 1 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0005 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0005 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 weight 1.0000 -> 0.8368 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12 weight 1.0000 -> 0.8609 2024-09-17T17:30:42.345 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13 weight 1.0000 -> 0.8942 2024-09-17T17:30:42.355 INFO:tasks.rados.rados.0.smithi102.stdout:348: writing smithi10238019-36 from 2015232 to 2162688 tid 1 2024-09-17T17:30:42.355 INFO:tasks.rados.rados.0.smithi102.stdout:349: rmattr oid 3 current snap is 30 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:349: done (3 left) 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 41 current snap is 30 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:345: finishing rollback tid 0 to smithi10238019-15 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:345: finishing rollback tid 1 to smithi10238019-15 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:345: finishing rollback tid 2 to smithi10238019-15 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 15 v 81 (ObjNum 122 snap 19 seq_num 122) dirty exists 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 21 v 58 (ObjNum 126 snap 19 seq_num 126) dirty exists 2024-09-17T17:30:42.356 INFO:tasks.rados.rados.0.smithi102.stdout:350: seq_num 153 ranges {1851392=163840} 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:350: writing smithi10238019-41 from 1851392 to 2015232 tid 1 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:345: done (3 left) 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:346: done (2 left) 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:351: copy_from oid 31 from oid 30 current snap is 30 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:352: rollback oid 4 current snap is 30 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 4 to 14 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:353: read oid 44 snap -1 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:353: expect deleted 2024-09-17T17:30:42.367 INFO:tasks.rados.rados.0.smithi102.stdout:354: delete oid 26 current snap is 30 2024-09-17T17:30:42.368 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T17:30:42.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T17:30:42.369 INFO:tasks.rados.rados.0.smithi102.stdout:348: finishing write tid 1 to smithi10238019-36 2024-09-17T17:30:42.369 INFO:tasks.rados.rados.0.smithi102.stdout:348: finishing write tid 2 to smithi10238019-36 2024-09-17T17:30:42.370 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:42.367+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:354: done (5 left) 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:355: read oid 38 snap -1 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:355: expect (ObjNum 125 snap 19 seq_num 125) 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:356: copy_from oid 9 from oid 6 current snap is 30 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:357: read oid 29 snap -1 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:357: expect (ObjNum 147 snap 29 seq_num 147) 2024-09-17T17:30:42.374 INFO:tasks.rados.rados.0.smithi102.stdout:358: snap_remove snap 25 2024-09-17T17:30:42.435 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=80, reqid=client.4566.0:274, at_version=42'13, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.725041828999998, 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003864363, 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707919+0000", 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708479+0000", 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055997299999999996 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708513+0000", 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4359000000000001e-05 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708524+0000", 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0346999999999999e-05 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708600+0000", 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999997e-05 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708619+0000", 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8490000000000001e-05 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711752+0000", 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031327600000000001 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711783+0000", 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1035999999999999e-05 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=81, reqid=client.4566.0:275, at_version=42'14, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:42.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.722674024, 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025720560000000001, 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710286+0000", 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710287+0000", 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3300000000000001e-07 2024-09-17T17:30:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710492+0000", 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020486500000000001 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710535+0000", 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3510999999999997e-05 2024-09-17T17:30:42.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710545+0000", 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9869999999999994e-06 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710609+0000", 2024-09-17T17:30:42.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3379000000000004e-05 2024-09-17T17:30:42.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.710619+0000", 2024-09-17T17:30:42.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0098000000000001e-05 2024-09-17T17:30:42.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712823+0000", 2024-09-17T17:30:42.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022037350000000001 2024-09-17T17:30:42.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.712858+0000", 2024-09-17T17:30:42.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5648000000000002e-05 2024-09-17T17:30:42.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.449 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.6s6 42/35 ECSubWrite(tid=86, reqid=client.4566.0:282, at_version=42'30, trim_to=0'0, roll_forward_to=41'28))", 2024-09-17T17:30:42.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:42.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.699263254999998, 2024-09-17T17:30:42.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002415295, 2024-09-17T17:30:42.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:42.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733697+0000", 2024-09-17T17:30:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.733698+0000", 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2759999999999999e-06 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734149+0000", 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045118399999999998 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734186+0000", 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7109e-05 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734195+0000", 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0539999999999995e-06 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734270+0000", 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5165000000000003e-05 2024-09-17T17:30:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.734282+0000", 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2763e-05 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736069+0000", 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017863530000000001 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.736112+0000", 2024-09-17T17:30:42.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3390999999999998e-05 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.735393529, 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010715961, 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697586+0000", 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8556999999999999e-05 2024-09-17T17:30:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697629+0000", 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2982999999999998e-05 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697637+0000", 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697698+0000", 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0534e-05 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697710+0000", 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1745e-05 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708248+0000", 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010538476999999999 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708283+0000", 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4431999999999998e-05 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.732109403999999, 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074300950000000003, 2024-09-17T17:30:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:30:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700852+0000", 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1969999999999999e-06 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700867+0000", 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4960000000000001e-05 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700917+0000", 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0285999999999999e-05 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700931+0000", 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3332999999999999e-05 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701036+0000", 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105467 2024-09-17T17:30:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701054+0000", 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7827999999999999e-05 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708242+0000", 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071881929999999998 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708281+0000", 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8831000000000001e-05 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.731525863000002, 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068976910000000001, 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701435+0000", 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701447+0000", 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2424e-05 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1851e-05 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701527+0000", 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7069999999999992e-06 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701592+0000", 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4825000000000004e-05 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701600+0000", 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-06 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708305+0000", 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067049680000000004 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708332+0000", 2024-09-17T17:30:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6899000000000001e-05 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 48/32 ECSubWrite(tid=41, reqid=osd.8.0:40, at_version=0'0, trim_to=0'0, roll_forward_to=48'31))", 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.712736215, 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77214092400000001, 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720235+0000", 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0353999999999999e-05 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491642+0000", 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77140694700000001 2024-09-17T17:30:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491651+0000", 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491754+0000", 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010309 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491768+0000", 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144999999999999e-05 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492327+0000", 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000558998 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492365+0000", 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969000000000002e-05 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 51/35 ECSubWrite(tid=60, reqid=osd.12.0:60, at_version=51'16, trim_to=0'0, roll_forward_to=50'14))", 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.096953622999997, 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028553210000000001, 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336043+0000", 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6211000000000003e-05 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337026+0000", 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098293199999999995 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337033+0000", 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4919999999999996e-06 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337104+0000", 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0616000000000003e-05 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337116+0000", 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1454000000000001e-05 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338825+0000", 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001709824 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338862+0000", 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6477000000000003e-05 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.057870190999999, 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002547812, 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375090+0000", 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9299999999999999e-07 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375115+0000", 2024-09-17T17:30:42.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.427e-05 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375207+0000", 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2684999999999998e-05 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375215+0000", 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.824e-06 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375311+0000", 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6266000000000003e-05 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375323+0000", 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1557e-05 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377599+0000", 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022765390000000002 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377638+0000", 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8377999999999997e-05 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 57/32 ECSubWrite(tid=49, reqid=osd.8.0:49, at_version=57'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.056072264999997, 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026169420000000001, 2024-09-17T17:30:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2600000000000001e-07 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376909+0000", 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0905999999999999e-05 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376963+0000", 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.393e-05 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376970+0000", 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1130000000000003e-06 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377048+0000", 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8391000000000002e-05 2024-09-17T17:30:42.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377057+0000", 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379423+0000", 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023655939999999999 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379505+0000", 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2259e-05 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 57/32 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=57'44, trim_to=0'0, roll_forward_to=56'40))", 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.055911364000004, 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002350857, 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377049+0000", 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377068+0000", 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8235000000000001e-05 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377097+0000", 2024-09-17T17:30:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9501000000000001e-05 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377105+0000", 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377158+0000", 2024-09-17T17:30:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3390000000000002e-05 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377166+0000", 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2630000000000003e-06 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379364+0000", 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021974960000000002 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379400+0000", 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5992999999999999e-05 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.054616772999999, 2024-09-17T17:30:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028134560000000002, 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378344+0000", 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.47e-07 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378373+0000", 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.917e-05 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378410+0000", 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6259e-05 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378418+0000", 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5299999999999996e-06 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378488+0000", 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9487999999999996e-05 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378497+0000", 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3889999999999994e-06 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381121+0000", 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002624103 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381157+0000", 2024-09-17T17:30:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.557e-05 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 69/67 ECSubWrite(tid=131, reqid=client.4566.0:647, at_version=69'54, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.343609+0000", 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.089351384, 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002440715, 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.343609+0000", 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.343609+0000", 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.343610+0000", 2024-09-17T17:30:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0100000000000004e-07 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.343763+0000", 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015307400000000001 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.344363+0000", 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060064600000000004 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.344373+0000", 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2420000000000006e-06 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.344623+0000", 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025007600000000001 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.344632+0000", 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3279999999999995e-06 2024-09-17T17:30:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346006+0000", 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013742190000000001 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346049+0000", 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3328999999999998e-05 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 79/74 ECSubWrite(tid=247, reqid=client.4566.0:796, at_version=79'135, trim_to=0'0, roll_forward_to=78'133))", 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.058021438000001, 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027697429999999999, 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:30:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376108+0000", 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011690730000000001 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376132+0000", 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3760999999999998e-05 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376140+0000", 2024-09-17T17:30:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.191e-06 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376198+0000", 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7695000000000002e-05 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376205+0000", 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2089999999999999e-06 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.377683+0000", 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014781639999999999 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.377708+0000", 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5445000000000001e-05 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=272, reqid=client.4566.0:841, at_version=85'148, trim_to=0'0, roll_forward_to=83'146))", 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.069467371, 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028713852000000002, 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.364388+0000", 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089506500000000001 2024-09-17T17:30:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390674+0000", 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026285665999999999 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390684+0000", 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0424e-05 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390748+0000", 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4132000000000002e-05 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390756+0000", 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1249999999999993e-06 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.392176+0000", 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001419783 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.392207+0000", 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0375e-05 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=273, reqid=client.4566.0:843, at_version=85'149, trim_to=0'0, roll_forward_to=83'146))", 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.390583+0000", 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0423767059999998, 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048863159999999999, 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390583+0000", 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390583+0000", 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390585+0000", 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.246e-06 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390655+0000", 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9933999999999998e-05 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394610+0000", 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039550799999999997 2024-09-17T17:30:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394619+0000", 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2699999999999993e-06 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394677+0000", 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7744000000000001e-05 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394688+0000", 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1362999999999999e-05 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.395431+0000", 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074258799999999997 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.395470+0000", 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9091000000000002e-05 2024-09-17T17:30:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=274, reqid=client.4566.0:844, at_version=85'150, trim_to=0'0, roll_forward_to=85'148))", 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.394381+0000", 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0385787180000001, 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044568630000000001, 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394381+0000", 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394381+0000", 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394382+0000", 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5300000000000003e-07 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394589+0000", 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020663800000000001 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397623+0000", 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030343689999999999 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397629+0000", 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397681+0000", 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1746999999999998e-05 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397692+0000", 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1686e-05 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.398799+0000", 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001106671 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.398838+0000", 2024-09-17T17:30:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9362e-05 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=275, reqid=client.4566.0:845, at_version=85'151, trim_to=0'0, roll_forward_to=85'149))", 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.397416+0000", 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0355438360000004, 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052212270000000002, 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397416+0000", 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397416+0000", 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397417+0000", 2024-09-17T17:30:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397609+0000", 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000192146 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401693+0000", 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040836860000000004 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401702+0000", 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1160000000000005e-06 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401760+0000", 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7853999999999999e-05 2024-09-17T17:30:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401771+0000", 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0988999999999999e-05 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402611+0000", 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00084015999999999995 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402638+0000", 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6565999999999999e-05 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=277, reqid=client.4566.0:847, at_version=85'153, trim_to=0'0, roll_forward_to=85'150))", 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.402653+0000", 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0303071660000001, 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11847311000000001, 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402653+0000", 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402653+0000", 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402654+0000", 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.046e-06 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402670+0000", 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.641e-05 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520286+0000", 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.117615488 2024-09-17T17:30:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520302+0000", 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5724000000000001e-05 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520454+0000", 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015243400000000001 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520474+0000", 2024-09-17T17:30:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9877e-05 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.521094+0000", 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062014800000000005 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.521126+0000", 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1983000000000002e-05 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.089622812, 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031028610000000002, 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.48e-07 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343358+0000", 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0500999999999999e-05 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343402+0000", 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4209000000000003e-05 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343415+0000", 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3072e-05 2024-09-17T17:30:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343484+0000", 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8427000000000001e-05 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343497+0000", 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3077e-05 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.346406+0000", 2024-09-17T17:30:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029091080000000001 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.346440+0000", 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4319000000000001e-05 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.540 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:42.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:42.542 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:42.540+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:42.643 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:42.641+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:42.690 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:42.690 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:42.690 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:42.690 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:42.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:42.743 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:42.741+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:42.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:42.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:42.807 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:42.807 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:42.807 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:42.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:42.844 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:42.842+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:42.925 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:42.925 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:42.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:42.925 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 39/35 ECSubWrite(tid=17, reqid=client.4566.0:186, at_version=0'0, trim_to=0'0, roll_forward_to=39'12))", 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.237880881000002, 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002820571, 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:42.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684876+0000", 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684890+0000", 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3499000000000001e-05 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685111+0000", 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022111 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685121+0000", 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7070000000000004e-06 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685199+0000", 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8145999999999993e-05 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685209+0000", 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0138e-05 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687657+0000", 2024-09-17T17:30:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024481260000000001 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687696+0000", 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.909e-05 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=94, reqid=client.4566.0:406, at_version=51'30, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.579545539999998, 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003277556, 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343212+0000", 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343225+0000", 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414999999999999e-05 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343249+0000", 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3983999999999999e-05 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343254+0000", 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5249999999999996e-06 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343307+0000", 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2167999999999999e-05 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343315+0000", 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8580000000000006e-06 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346453+0000", 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003137908 2024-09-17T17:30:42.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346488+0000", 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5021999999999998e-05 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 51/35 ECSubWrite(tid=62, reqid=osd.12.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=51'30))", 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.578203324, 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025408480000000001, 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344553+0000", 2024-09-17T17:30:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344554+0000", 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8100000000000001e-07 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344567+0000", 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.254e-05 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344609+0000", 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2549000000000002e-05 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344619+0000", 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6299999999999993e-06 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344683+0000", 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4090999999999993e-05 2024-09-17T17:30:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.344693+0000", 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0363000000000001e-05 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347044+0000", 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023509339999999998 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.347094+0000", 2024-09-17T17:30:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.986e-05 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.533238881999999, 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024942340000000001, 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389518+0000", 2024-09-17T17:30:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389963+0000", 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044482000000000001 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389993+0000", 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.995e-05 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.389999+0000", 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7830000000000004e-06 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390051+0000", 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2475999999999997e-05 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.390057+0000", 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7540000000000003e-06 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.391979+0000", 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001921835 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.392012+0000", 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2848000000000001e-05 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 56/35 ECSubWrite(tid=57, reqid=osd.10.0:57, at_version=56'41, trim_to=0'0, roll_forward_to=53'39))", 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.587492965000003, 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036979209999999998, 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335264+0000", 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335298+0000", 2024-09-17T17:30:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3837000000000001e-05 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335654+0000", 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035630699999999999 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335665+0000", 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1620999999999999e-05 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335751+0000", 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5692999999999994e-05 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335763+0000", 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140000000000001e-05 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338926+0000", 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031629399999999999 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338961+0000", 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5129999999999997e-05 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.584687250999998, 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003185701, 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338094+0000", 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4428000000000001e-05 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011644700000000001 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338219+0000", 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9020000000000005e-06 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338416+0000", 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196558 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338424+0000", 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.585e-06 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341209+0000", 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027847050000000002 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341255+0000", 2024-09-17T17:30:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.584646020999998, 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004124801, 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:30:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9600000000000001e-07 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338129+0000", 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8003e-05 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338243+0000", 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114367 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338249+0000", 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3160000000000002e-06 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338788+0000", 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053876399999999995 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338797+0000", 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-06 2024-09-17T17:30:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342196+0000", 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033993830000000002 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342235+0000", 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8921999999999998e-05 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.584547086999997, 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040538249999999996, 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.99e-07 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338233+0000", 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3623999999999999e-05 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338275+0000", 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2249000000000002e-05 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338284+0000", 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2609999999999993e-06 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338436+0000", 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152032 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338448+0000", 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2469999999999999e-05 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342230+0000", 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037816899999999999 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342263+0000", 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3300000000000003e-05 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.584486476000002, 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029941270000000001, 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338289+0000", 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8340000000000001e-05 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338322+0000", 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3419e-05 2024-09-17T17:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338332+0000", 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0086e-05 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338873+0000", 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054120000000000004 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338882+0000", 2024-09-17T17:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1329999999999999e-06 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341233+0000", 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023510699999999998 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1365999999999998e-05 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.550381643999998, 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285746, 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372399+0000", 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4416000000000001e-05 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372479+0000", 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9556999999999997e-05 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372487+0000", 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7589999999999997e-06 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372561+0000", 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3807000000000007e-05 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372571+0000", 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0532e-05 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375622+0000", 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030507860000000002 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375661+0000", 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8634000000000003e-05 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.549651877999999, 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004179131, 2024-09-17T17:30:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373125+0000", 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9626000000000001e-05 2024-09-17T17:30:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373159+0000", 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4190999999999998e-05 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373168+0000", 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6999999999999997e-06 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373237+0000", 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9306000000000004e-05 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373246+0000", 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377245+0000", 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039987390000000003 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377284+0000", 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8423999999999997e-05 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.549305760000003, 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048101309999999996, 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1800000000000005e-07 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373476+0000", 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4712999999999999e-05 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373535+0000", 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8536999999999999e-05 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373545+0000", 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0142e-05 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374163+0000", 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061771899999999997 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374172+0000", 2024-09-17T17:30:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.081e-06 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378233+0000", 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040611509999999998 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378261+0000", 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000002e-05 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.549255674999998, 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038195389999999998, 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373518+0000", 2024-09-17T17:30:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373595+0000", 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6206999999999997e-05 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373601+0000", 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6039999999999998e-06 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373659+0000", 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7442000000000003e-05 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373668+0000", 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2040000000000003e-06 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377300+0000", 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00363182 2024-09-17T17:30:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377320+0000", 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0598000000000002e-05 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.546654150000002, 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034371530000000001, 2024-09-17T17:30:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376103+0000", 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376128+0000", 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5109e-05 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376199+0000", 2024-09-17T17:30:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1783e-05 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376209+0000", 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1710000000000008e-06 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376272+0000", 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749000000000002e-05 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376281+0000", 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379498+0000", 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003216605 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379539+0000", 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1587e-05 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 64/35 ECSubWrite(tid=108, reqid=client.4566.0:565, at_version=64'40, trim_to=0'0, roll_forward_to=60'38))", 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.585284758, 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002766442, 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337472+0000", 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337646+0000", 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017400099999999999 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337696+0000", 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0402999999999998e-05 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337712+0000", 2024-09-17T17:30:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5441999999999999e-05 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337788+0000", 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6283000000000001e-05 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.337842+0000", 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3681999999999999e-05 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.340201+0000", 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023587859999999999 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:15.340238+0000", 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7404999999999998e-05 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 68/67 ECSubWrite(tid=75, reqid=client.4566.0:644, at_version=68'52, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.222809570999999, 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52365443700000003, 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699948+0000", 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999998e-07 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699964+0000", 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6784000000000001e-05 2024-09-17T17:30:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222607+0000", 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52264301099999999 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222621+0000", 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3923e-05 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222745+0000", 2024-09-17T17:30:42.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012366800000000001 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222753+0000", 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6490000000000008e-06 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223566+0000", 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081285500000000002 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:42.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223601+0000", 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5744000000000002e-05 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/74 ECSubWrite(tid=163, reqid=osd.6.0:163, at_version=75'29, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.571694599000001, 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.34608951199999999, 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351063+0000", 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9999999999999997e-07 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351085+0000", 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2129000000000001e-05 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696028+0000", 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.344942945 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696043+0000", 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5353e-05 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696175+0000", 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131996 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696189+0000", 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3573e-05 2024-09-17T17:30:43.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697107+0000", 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091841899999999996 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697151+0000", 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4397000000000002e-05 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.13s4 80/74 ECSubWrite(tid=111, reqid=osd.10.0:111, at_version=80'62, trim_to=0'0, roll_forward_to=79'60))", 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.575336026, 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003050119, 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347421+0000", 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.03e-07 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347462+0000", 2024-09-17T17:30:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0862999999999998e-05 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347534+0000", 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2643000000000002e-05 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347550+0000", 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5633000000000001e-05 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347706+0000", 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015574400000000001 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347737+0000", 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1164000000000002e-05 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350445+0000", 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027080540000000001 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350470+0000", 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5715e-05 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.346955+0000", 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.57580156199999999, 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002824015, 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.346955+0000", 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.346955+0000", 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.346955+0000", 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5400000000000001e-07 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.346975+0000", 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9726000000000001e-05 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.347001+0000", 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5789e-05 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.347008+0000", 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.347834+0000", 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000826047 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.347844+0000", 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0581000000000001e-05 2024-09-17T17:30:43.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349756+0000", 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019120529999999999 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349779+0000", 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.249e-05 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 91/76 ECSubWrite(tid=201, reqid=osd.12.0:201, at_version=91'65, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.349095+0000", 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.57366096, 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027285209999999998, 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349095+0000", 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349095+0000", 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349096+0000", 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0900000000000001e-07 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349117+0000", 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1254e-05 2024-09-17T17:30:43.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349156+0000", 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9230000000000002e-05 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349164+0000", 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8169999999999999e-06 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349925+0000", 2024-09-17T17:30:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076142799999999997 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.349934+0000", 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8429999999999999e-06 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.351791+0000", 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018570769999999999 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.351824+0000", 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2663000000000002e-05 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:43.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:43.020 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:43.018+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:43.120 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:43.118+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:43.126 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:43.126 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:43.126 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:43.126 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:43.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:43.221 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:43.220+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:43.242 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:43.242 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:43.242 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:43.242 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:43.242 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:43.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:43.323 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:43.321+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:43.344 INFO:tasks.rados.rados.0.smithi102.stdout:358: done (8 left) 2024-09-17T17:30:43.344 INFO:tasks.rados.rados.0.smithi102.stdout:359: snap_remove snap 28 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:moved 5 / 200 (2.5%) 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:avg 12.5 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 2.78388 -> 2.62202 (expected baseline 3.42327) 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.12 with 6 -> 6 pgs (0.48 -> 0.48 * mean) 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.1 with 16 -> 14 pgs (1.28 -> 1.12 * mean) 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:oload 110 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 1 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0005 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0005 2024-09-17T17:30:43.346 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 weight 0.8368 -> 0.6865 2024-09-17T17:30:43.347 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12 weight 0.8609 -> 0.7424 2024-09-17T17:30:43.347 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13 weight 0.8942 -> 0.8009 2024-09-17T17:30:43.368 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T17:30:43.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-17T17:30:43.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.723958015999997, 2024-09-17T17:30:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:30:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.710421846999999, 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.708195988, 2024-09-17T17:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.707771909000002, 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.707677607000001, 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=82, reqid=osd.3.0:82, at_version=57'61, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.037339752999998, 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035036519999999999, 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-07 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371301+0000", 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6461e-05 2024-09-17T17:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372066+0000", 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076444900000000005 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372076+0000", 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372145+0000", 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9586999999999996e-05 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372155+0000", 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0054999999999999e-05 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374734+0000", 2024-09-17T17:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790520000000001 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374778+0000", 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4042000000000003e-05 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=57'63, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.035250456, 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026238189999999999, 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373365+0000", 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0640000000000001e-06 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373399+0000", 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3757000000000002e-05 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373439+0000", 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0028999999999999e-05 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373469+0000", 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0297000000000001e-05 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373543+0000", 2024-09-17T17:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4202999999999994e-05 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373574+0000", 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0895999999999997e-05 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375923+0000", 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023489689999999998 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375988+0000", 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4603999999999993e-05 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:581 3.es0 3:7a426564:::smithi10238019-31:head [create,setxattr TahZbq^WXaahVeQfQsXqNXgcd (1963) in=25b,setxattr `Pj`q[kZqXoPYYj_`hNV^bohgce[Ull^lRXTctln (982) in=40b,setxattr ^gfWYcnTRmaWZQkrf`asZ\\XVPcRmRdRckrhUheci (10312) in=40b,setxattr hVm[Q^c]jmYfcedO_eoe[N_fohZ^nghUPgNQYY`h (15648) in=40b,setxattr \\jlYtrYeh_rthhUaqacc]`Uger\\]rPngbjPhbaWV (454) in=40b,setxattr nOgORnqOheegXnS]tSkTQTreXqqt`ojnVPsQbabf (771) in=40b,setxattr QR]c[o[]SToV\\goc\\pjtRRf`WYSngR[eQ`taQgcc (12488) in=40b,setxattr pY\\tNq\\a[n\\fajnSVYNng[\\kcqZ^VhqNmqRXUTlO (1121) in=40b,setxattr [laaiUa\\cY`kdXP\\SNRq[ig]rcanlRgnTtdfZi_W (1403) in=40b,setxattr aaf[PNUURnfSbT^fR_ge[lfbq]keQjYgrbO\\SbTa (1226) in=40b,setxattr SfaWR`YpWpbnn]sR]^eO[Wbi\\Se_TkrcQp[pcomR (4768) in=40b,setxattr pS^PVQeeOqs_S^YVlb^n\\h``m_Q\\prek^_qjcmgR (1121) in=40b,setxattr [aWfmed^jZheSjc^Xmb]TrYaU[[c]jb\\[UdNQngi (593) in=40b,setxattr kPO^emkeWSaQtnWaRVXk[hbN[TP^]ThcqpOfordV (2805) in=40b,setxattr PkhW`mV]l_O[nUSprX^tRPoRakfUNbOiRedpZb^n (421) in=40b,setxattr [`esP (1963) in=5b,setxattr SlhkjUdadks\\hNrmRohr_fl`itUjb_Vjia\\Vdqf` (10033) in=40b,setxattr TUWTXRdZcQc\\lOTbYmrWiirilafTQQrNdPoj_^sZ (3084) in=40b,setxattr ddbfaqshSg (1963) in=10b,setxattr nSgahmseaQO^hNUcamf\\m[SsQam^Uc``at``i`kU (1435) in=40b] snapc 11=[11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.046747251999999, 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047340350000000002, 2024-09-17T17:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 581 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361869+0000", 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.623e-06 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362010+0000", 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014150499999999999 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362027+0000", 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6569e-05 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362040+0000", 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2767e-05 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362112+0000", 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2401e-05 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362122+0000", 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0013000000000001e-05 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.363809+0000", 2024-09-17T17:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016863329999999999 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.363887+0000", 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8323000000000002e-05 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.364760+0000", 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087325800000000002 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.365703+0000", 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094332599999999997 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.366501+0000", 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000797326 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.366601+0000", 2024-09-17T17:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100591 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:656 3.es0 3:7a426564:::smithi10238019-31:head [append 0~0] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.061098167000001, 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081233910000000006, 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 656 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347519+0000", 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6850000000000001e-06 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347531+0000", 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2339e-05 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347553+0000", 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2005999999999999e-05 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347564+0000", 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0944999999999999e-05 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347643+0000", 2024-09-17T17:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8885000000000001e-05 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347652+0000", 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350793+0000", 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031410100000000001 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350899+0000", 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105824 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.351778+0000", 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087886400000000003 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353482+0000", 2024-09-17T17:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001703577 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355585+0000", 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021028700000000002 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355640+0000", 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5186999999999999e-05 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:657 3.es0 3:7a426564:::smithi10238019-31:head [rollback f] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.061018881999999, 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088160730000000007, 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 657 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7800000000000001e-07 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347600+0000", 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.501e-06 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347608+0000", 2024-09-17T17:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5990000000000002e-06 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347615+0000", 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347721+0000", 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106421 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347727+0000", 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0349999999999998e-06 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.352452+0000", 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047242070000000002 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.352548+0000", 2024-09-17T17:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5958000000000005e-05 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353151+0000", 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060365899999999999 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354055+0000", 2024-09-17T17:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090400699999999997 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356330+0000", 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022750029999999998 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356412+0000", 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1454999999999999e-05 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 69/32 ECSubWrite(tid=126, reqid=client.4566.0:661, at_version=69'95, trim_to=0'0, roll_forward_to=69'93))", 2024-09-17T17:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.367835+0000", 2024-09-17T17:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.040779085, 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023874859999999999, 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.367835+0000", 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.367835+0000", 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.367836+0000", 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5899999999999996e-07 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368759+0000", 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000922532 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368785+0000", 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6265999999999999e-05 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368790+0000", 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3020000000000002e-06 2024-09-17T17:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368839+0000", 2024-09-17T17:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9098000000000003e-05 2024-09-17T17:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.368847+0000", 2024-09-17T17:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2389999999999996e-06 2024-09-17T17:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.370196+0000", 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013490640000000001 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.370223+0000", 2024-09-17T17:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7325999999999999e-05 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:726 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~606208] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.057011754000001, 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049411350000000001, 2024-09-17T17:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 726 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351604+0000", 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6700000000000001e-06 2024-09-17T17:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352206+0000", 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060191499999999996 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352220+0000", 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.342e-05 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352227+0000", 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5989999999999996e-06 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352307+0000", 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9568999999999997e-05 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352377+0000", 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9842e-05 2024-09-17T17:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.353455+0000", 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001077801 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354825+0000", 2024-09-17T17:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001370509 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356461+0000", 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016354329999999999 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356544+0000", 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3376999999999998e-05 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:727 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~507904] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.054045296000002, 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027356989999999999, 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 727 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354571+0000", 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.302e-06 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355037+0000", 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046691300000000001 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355070+0000", 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2064000000000003e-05 2024-09-17T17:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355079+0000", 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8260000000000005e-06 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355121+0000", 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1813000000000001e-05 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355196+0000", 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4367000000000001e-05 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355749+0000", 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055320300000000005 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356631+0000", 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00088253400000000003 2024-09-17T17:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357264+0000", 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063273500000000005 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357305+0000", 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0942000000000002e-05 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:728 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~573440] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.050808055000001, 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028748519999999998, 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 728 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357808+0000", 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3650000000000001e-06 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358354+0000", 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054612300000000003 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358365+0000", 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1242e-05 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358371+0000", 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5219999999999998e-06 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358404+0000", 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139000000000002e-05 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358466+0000", 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2359000000000004e-05 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358974+0000", 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050814800000000004 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.359801+0000", 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082624900000000004 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360638+0000", 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083704500000000002 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360681+0000", 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3659999999999999e-05 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:729 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~475136] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.048105111999998, 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034332189999999999, 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 729 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360510+0000", 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0699999999999996e-07 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360948+0000", 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000437978 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360962+0000", 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3478999999999999e-05 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360969+0000", 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4680000000000003e-06 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361023+0000", 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3381999999999998e-05 2024-09-17T17:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361088+0000", 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5579000000000004e-05 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361618+0000", 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052954599999999999 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362780+0000", 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001162787 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363897+0000", 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011169929999999999 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363943+0000", 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000001e-05 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:730 3.es0 3:74f6cb56:::smithi10238019-48:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.047627578, 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030870279999999999, 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 730 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360988+0000", 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.850000000000001e-07 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360995+0000", 2024-09-17T17:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5120000000000002e-06 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361004+0000", 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361013+0000", 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4440000000000008e-06 2024-09-17T17:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362000+0000", 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098763200000000001 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362055+0000", 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4580000000000003e-05 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362399+0000", 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034424899999999998 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363048+0000", 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00064926700000000003 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364011+0000", 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096274900000000005 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364074+0000", 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2903000000000004e-05 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:731 3.es0 3:74f6cb56:::smithi10238019-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.047589533, 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039696050000000002, 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 731 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361026+0000", 2024-09-17T17:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361028+0000", 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.678e-06 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361035+0000", 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9829999999999999e-06 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361041+0000", 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362876+0000", 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001835267 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362939+0000", 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2508000000000005e-05 2024-09-17T17:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364285+0000", 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013463780000000001 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364367+0000", 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2318999999999996e-05 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364995+0000", 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000627182 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:7 3.es0 3:71a2b96a:::smithi10238019-23:head [assert-version v25,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.047366845000001, 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077714619999999998, 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361249+0000", 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.579e-06 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361265+0000", 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5812999999999999e-05 2024-09-17T17:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361290+0000", 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4666000000000001e-05 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361298+0000", 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1990000000000006e-06 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361353+0000", 2024-09-17T17:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4898999999999997e-05 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361425+0000", 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1644999999999996e-05 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.369019+0000", 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075946609999999999 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:895 3.es0 3:7a426564:::smithi10238019-31:head [copy-from ver 68] snapc 1f=[1f,1e,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.365975+0000", 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0426400469999999, 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024857400000000002, 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 895 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365975+0000", 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365975+0000", 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365976+0000", 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6500000000000001e-06 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365984+0000", 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365997+0000", 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3365e-05 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366004+0000", 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4850000000000003e-06 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366071+0000", 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6552999999999998e-05 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366176+0000", 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010516 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383179+0000", 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017002529999999998 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385694+0000", 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025152289999999999 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.390726+0000", 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050323570000000003 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.390832+0000", 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105641 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:896 3.es0 3:7a426564:::smithi10238019-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.366029+0000", 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.042585667, 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019356549000000001, 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 896 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366029+0000", 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366029+0000", 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366030+0000", 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366033+0000", 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.247e-06 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366041+0000", 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3960000000000006e-06 2024-09-17T17:30:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366048+0000", 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0949999999999998e-06 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366332+0000", 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028421099999999998 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:43.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366363+0000", 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0908999999999998e-05 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385136+0000", 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018772481000000001 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385238+0000", 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101969 2024-09-17T17:30:43.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385385+0000", 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014762999999999999 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.536 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:43.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:43.539 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:43.536+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:43.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:43.639 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:43.637+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:43.687 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:43.688 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:43.688 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:43.688 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:43.689 INFO:teuthology.orchestra.run.smithi007.stdout:moved 0 / 200 (0%) 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:avg 12.5 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 2.62202 -> 2.62202 (expected baseline 3.42327) 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.12 with 6 -> 6 pgs (0.48 -> 0.48 * mean) 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.2 with 16 -> 16 pgs (1.28 -> 1.28 * mean) 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:oload 1000 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 0.05 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0005 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0049 2024-09-17T17:30:43.690 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T17:30:43.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:43.751 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:43.749+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:43.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:43.805 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:43.806 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:43.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:43.807 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:43.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:43.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:43.828 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:43.841 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:43.838+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:43.849 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:43.849 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:43.870 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:43.914 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:43.936 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:43.937 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.2516064, 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.220653177000003, 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.188351533999999, 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:30:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:30:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:30:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:30:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.186427031999997, 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:30:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:410 3.9s0 3:93d1baed:::smithi10238019-5:head [copy-from ver 8] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.596530838, 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017989010999999999, 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 410 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338102+0000", 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5349999999999999e-06 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338107+0000", 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000003e-06 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338121+0000", 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3743e-05 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338129+0000", 2024-09-17T17:30:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4209999999999995e-06 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8885000000000003e-05 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338278+0000", 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0192999999999995e-05 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.349495+0000", 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011217173 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350828+0000", 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013328700000000001 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356035+0000", 2024-09-17T17:30:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052068000000000001 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356090+0000", 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4731000000000002e-05 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 52/35 ECSubWriteReply(tid=76, last_complete=52'24, committed=1, applied=1))", 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.525079214000002, 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.18030790999999999, 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409553+0000", 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409560+0000", 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589756+0000", 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.180196197 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589765+0000", 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589834+0000", 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9301000000000006e-05 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589860+0000", 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6058999999999999e-05 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:479 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~131072] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.586553690000002, 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017167436000000001, 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 479 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348079+0000", 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.719e-06 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348181+0000", 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101258 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348196+0000", 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5423000000000001e-05 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348204+0000", 2024-09-17T17:30:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348898+0000", 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069432499999999998 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349042+0000", 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143848 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349752+0000", 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071013599999999999 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351054+0000", 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301618 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365180+0000", 2024-09-17T17:30:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014125511 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365245+0000", 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5647000000000004e-05 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:480 3.0s0 3:00c2d3b1:::smithi10238019-29:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.586411322000004, 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017175950999999998, 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 480 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348221+0000", 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348225+0000", 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969999999999998e-06 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348235+0000", 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312000000000001e-05 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348241+0000", 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340000000000002e-06 2024-09-17T17:30:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350222+0000", 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019816209999999998 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350336+0000", 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113779 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350754+0000", 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041740400000000002 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351686+0000", 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093241600000000004 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365336+0000", 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013650254000000001 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365396+0000", 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9704000000000003e-05 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:481 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.586380239999997, 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018899091, 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 481 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348252+0000", 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348255+0000", 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6720000000000002e-06 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348265+0000", 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.132e-06 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348271+0000", 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351236+0000", 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029650309999999999 2024-09-17T17:30:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351333+0000", 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7449999999999997e-05 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365440+0000", 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106966 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365519+0000", 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8703999999999997e-05 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.367150+0000", 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001631354 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 68/32 ECSubWrite(tid=191, reqid=client.4566.0:626, at_version=68'97, trim_to=0'0, roll_forward_to=68'94))", 2024-09-17T17:30:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.434385258999999, 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018870181999999999, 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500247+0000", 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8299999999999996e-07 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500279+0000", 2024-09-17T17:30:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2601000000000002e-05 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518513+0000", 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018233269 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518520+0000", 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2590000000000003e-06 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518590+0000", 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518600+0000", 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7329999999999998e-06 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519071+0000", 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047090800000000003 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519116+0000", 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5521e-05 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.12s1 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.433944224000001, 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84719745400000002, 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500688+0000", 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3e-07 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500724+0000", 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-05 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346201+0000", 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84547703600000002 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346210+0000", 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346289+0000", 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8825999999999994e-05 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346300+0000", 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1602e-05 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347848+0000", 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00154776 2024-09-17T17:30:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347885+0000", 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6723000000000001e-05 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:651 3.0s0 3:07ceeeba:::smithi10238019-36:head [setxattr _header (57) in=7b] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.587323343000001, 2024-09-17T17:30:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.090354326999999998, 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 651 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347309+0000", 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:30:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347313+0000", 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7270000000000001e-06 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347320+0000", 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1019999999999999e-06 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347330+0000", 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0552e-05 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353490+0000", 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061600309999999998 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353716+0000", 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000225987 2024-09-17T17:30:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354187+0000", 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047076600000000001 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355021+0000", 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083427900000000005 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437590+0000", 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.082568902999999999 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437662+0000", 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2272000000000002e-05 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:652 3.0s0 3:07ceeeba:::smithi10238019-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.587291293, 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.091356190000000004, 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 652 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347341+0000", 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347344+0000", 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0000000000000001e-06 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347352+0000", 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8250000000000005e-06 2024-09-17T17:30:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347357+0000", 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6489999999999996e-06 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354533+0000", 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007175781 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354625+0000", 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2064999999999994e-05 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438014+0000", 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.083388730999999994 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438102+0000", 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7861000000000005e-05 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438696+0000", 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059467799999999996 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 69/67 ECSubWriteReply(tid=135, last_complete=69'58, committed=1, applied=1))", 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.578502386, 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081843374999999996, 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356130+0000", 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3600000000000003e-07 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356133+0000", 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.794e-06 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437473+0000", 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081339488000000001 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437480+0000", 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6810000000000001e-06 2024-09-17T17:30:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437533+0000", 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2958999999999999e-05 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437972+0000", 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043981700000000001 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 73/72 ECSubReadReply(tid=170, attrs_read=0))", 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.901698691, 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.308285174, 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.37e-07 2024-09-17T17:30:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.033151+0000", 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000217491 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337311+0000", 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.304160612 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337319+0000", 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4030000000000001e-06 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.340881+0000", 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035619649999999998 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.341218+0000", 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033716600000000002 2024-09-17T17:30:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 74/72 ECSubReadReply(tid=172, attrs_read=0))", 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.579555257999999, 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64532973699999996, 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355077+0000", 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355304+0000", 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000086+0000", 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64478191900000004 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000092+0000", 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1800000000000001e-06 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000163+0000", 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0883999999999994e-05 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000406+0000", 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024273599999999999 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPushReply(3.9s0 74/72 [PushReplyOp(3:96cbaefd:::smithi10238019-46:head)])", 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.922222616999999, 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32756439999999998, 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012412+0000", 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7250000000000002e-06 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337269+0000", 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32485740099999999 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337283+0000", 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3655999999999999e-05 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339351+0000", 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020684850000000001 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339973+0000", 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062179200000000003 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 89/75 ECSubWriteReply(tid=200, last_complete=89'71, committed=1, applied=1))", 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.2158856509999998, 2024-09-17T17:30:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91509105899999998, 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718747+0000", 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.243e-06 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633701+0000", 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91495358800000004 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633709+0000", 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633787+0000", 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7503999999999995e-05 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633837+0000", 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0012000000000002e-05 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:891 3.0s0 3:07ceeeba:::smithi10238019-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.354567+0000", 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.580064414, 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015286357, 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 891 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.354567+0000", 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.354567+0000", 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.354567+0000", 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.354570+0000", 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4700000000000001e-06 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.354577+0000", 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3000000000000004e-06 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.354583+0000", 2024-09-17T17:30:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6049999999999997e-06 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.357864+0000", 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032813759999999999 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.357965+0000", 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101063 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.369219+0000", 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011253912 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.369327+0000", 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107357 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.369853+0000", 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052678100000000004 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:911 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 822626~1028766,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.5618884550000001, 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.94643117499999996, 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 911 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372744+0000", 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6000000000000003e-07 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372746+0000", 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372758+0000", 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999e-05 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372764+0000", 2024-09-17T17:30:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5289999999999999e-06 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.373482+0000", 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071808600000000005 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.373547+0000", 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5382000000000005e-05 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.319174+0000", 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.94562681400000004 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:44.042 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:44.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:44.044 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:44.045 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:44.045 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:44.045 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:44.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:44.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:44.047 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:44.041+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:44.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:44.085+0000 7f4d857e2640 0 -- 172.21.15.7:0/2791084125 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f4d80221680 msgr2=0x7f4d80223a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:44.115 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:44.115 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:44.167 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.167 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.167 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:44.165+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:44.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:44.244 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:44.242+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:44.283 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.283 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:30:44.345 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:44.343+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:44.347 INFO:tasks.rados.rados.0.smithi102.stdout:350: finishing write tid 1 to smithi10238019-41 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:350: finishing write tid 2 to smithi10238019-41 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:352: finishing rollback tid 0 to smithi10238019-4 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:352: finishing rollback tid 1 to smithi10238019-4 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:350: finishing write tid 3 to smithi10238019-41 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:350: oid 41 updating version 0 to 28 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:350: oid 41 version 28 is already newer than 27 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 41 v 28 (ObjNum 153 snap 30 seq_num 153) dirty exists 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:350: left oid 41 (ObjNum 153 snap 30 seq_num 153) 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:356: finishing copy_from to smithi10238019-9 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:356: got expected ENOENT (src dne) 2024-09-17T17:30:44.348 INFO:tasks.rados.rados.0.smithi102.stdout:356: finishing copy_from racing read to smithi10238019-9 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:348: finishing write tid 3 to smithi10238019-36 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:348: oid 36 updating version 0 to 75 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:348: oid 36 updating version 75 to 76 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 36 v 76 (ObjNum 152 snap 30 seq_num 152) dirty exists 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:348: left oid 36 (ObjNum 152 snap 30 seq_num 152) 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:352: finishing rollback tid 2 to smithi10238019-4 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 4 v 32 (ObjNum 53 snap 2 seq_num 53) dirty exists 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:351: finishing copy_from racing read to smithi10238019-31 2024-09-17T17:30:44.349 INFO:tasks.rados.rados.0.smithi102.stdout:355: expect (ObjNum 125 snap 19 seq_num 125) 2024-09-17T17:30:44.355 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 886094 2024-09-17T17:30:44.370 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:44.392 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:44.410 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.411 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.413 INFO:tasks.rados.rados.0.smithi102.stdout:351: finishing copy_from to smithi10238019-31 2024-09-17T17:30:44.413 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 68 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-09-17T17:30:44.414 INFO:tasks.rados.rados.0.smithi102.stdout:357: expect (ObjNum 147 snap 29 seq_num 147) 2024-09-17T17:30:44.414 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:44.420 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:44.442 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:44.445 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:44.443+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:44.476 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 822626 2024-09-17T17:30:44.495 INFO:tasks.rados.rados.0.smithi102.stdout:348: done (8 left) 2024-09-17T17:30:44.495 INFO:tasks.rados.rados.0.smithi102.stdout:350: done (7 left) 2024-09-17T17:30:44.495 INFO:tasks.rados.rados.0.smithi102.stdout:351: done (6 left) 2024-09-17T17:30:44.495 INFO:tasks.rados.rados.0.smithi102.stdout:352: done (5 left) 2024-09-17T17:30:44.495 INFO:tasks.rados.rados.0.smithi102.stdout:353: done (4 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:355: done (3 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:356: done (2 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:357: done (1 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:359: done (0 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:360: rmattr oid 40 current snap is 30 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:360: done (0 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:361: rmattr oid 32 current snap is 30 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:361: done (0 left) 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:362: rollback oid 31 current snap is 30 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 31 to 22 2024-09-17T17:30:44.496 INFO:tasks.rados.rados.0.smithi102.stdout:363: snap_create 2024-09-17T17:30:44.497 INFO:tasks.rados.rados.0.smithi102.stdout:362: finishing rollback tid 0 to smithi10238019-31 2024-09-17T17:30:44.502 INFO:tasks.rados.rados.0.smithi102.stdout:362: finishing rollback tid 1 to smithi10238019-31 2024-09-17T17:30:44.502 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 31 v 70 (ObjNum 721 snap 0 seq_num 3463993832) dirty dne 2024-09-17T17:30:44.546 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:44.544+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:44.569 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.569 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:44.647 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:44.646+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:44.684 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.684 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:44.749 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:44.747+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:44.799 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.799 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:44.850 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:44.848+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:44.916 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:44.916 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:44.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:44.951 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:44.949+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:45.030 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.030 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:45.051 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:45.050+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:45.147 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.147 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.153 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:45.151+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:45.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:45.253 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:45.252+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:45.306 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.306 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:45.342 INFO:tasks.rados.rados.0.smithi102.stdout:362: done (1 left) 2024-09-17T17:30:45.342 INFO:tasks.rados.rados.0.smithi102.stdout:363: done (0 left) 2024-09-17T17:30:45.342 INFO:tasks.rados.rados.0.smithi102.stdout:364: delete oid 13 current snap is 31 2024-09-17T17:30:45.354 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:45.352+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:45.435 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.435 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:45.488 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:45.486+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:45.551 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.551 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.556 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:45.555+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:45.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:45.658 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:45.656+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:45.672 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.672 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:45.759 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:45.757+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:45.791 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.791 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:45.861 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:45.859+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:45.907 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:45.907 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:45.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:45.961 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:45.959+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:46.022 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:46.023 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:46.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:46.062 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:46.061+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:46.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:46.137 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:46.138 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:46.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:46.164 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:46.162+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:46.254 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:46.254 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:46.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:46.265 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:46.263+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:46.273 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:46.297 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:46.297 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:46.324 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:46.324 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:46.353 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:46.353 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:46.362 INFO:tasks.rados.rados.0.smithi102.stdout:364: done (0 left) 2024-09-17T17:30:46.362 INFO:tasks.rados.rados.0.smithi102.stdout:365: read oid 31 snap 21 2024-09-17T17:30:46.362 INFO:tasks.rados.rados.0.smithi102.stdout:365: expect deleted 2024-09-17T17:30:46.362 INFO:tasks.rados.rados.0.smithi102.stdout:366: snap_remove snap 29 2024-09-17T17:30:46.365 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:46.363+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:46.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:30:46.375 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:46.379 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:46.379 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:46.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:46.402 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:46.466 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:46.464+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:46.495 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:46.495 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:46.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:46.528 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:46.550 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:46.567 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:46.565+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:46.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:46.593 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:46.613 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:46.635 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:218 3.3s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.928302078999998, 2024-09-17T17:30:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65573700899999998, 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 218 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721616+0000", 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4e-07 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721619+0000", 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2030000000000001e-06 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721626+0000", 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721631+0000", 2024-09-17T17:30:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4539999999999998e-06 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722624+0000", 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00099297500000000011 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722675+0000", 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0627000000000002e-05 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722943+0000", 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026790700000000001 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723463+0000", 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000520003 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377270+0000", 2024-09-17T17:30:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65380684099999997 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377352+0000", 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2307999999999999e-05 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:219 3.3s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.928275855000003, 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65649164999999998, 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 219 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721642+0000", 2024-09-17T17:30:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0100000000000005e-07 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721644+0000", 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4719999999999998e-06 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721651+0000", 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.781e-06 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721656+0000", 2024-09-17T17:30:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4809999999999997e-06 2024-09-17T17:30:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723147+0000", 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001490587 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723197+0000", 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.003e-05 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377653+0000", 2024-09-17T17:30:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65445650300000002 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377716+0000", 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2223000000000004e-05 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.378133+0000", 2024-09-17T17:30:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041707199999999999 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:440 3.3s0 3:c80ea1ca:::smithi10238019-27:head [append 0~81920] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 43.245542333000003, 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19940718800000001, 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 440 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8299999999999996e-07 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404418+0000", 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2608000000000003e-05 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404437+0000", 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8810000000000001e-05 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404442+0000", 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.327e-06 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406246+0000", 2024-09-17T17:30:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018034769999999999 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406352+0000", 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106269 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406784+0000", 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00043238800000000002 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.408111+0000", 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001326502 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603719+0000", 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.195608474 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603782+0000", 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2550000000000003e-05 2024-09-17T17:30:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/35 ECSubWriteReply(tid=48, last_complete=52'36, committed=1, applied=1))", 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 43.242147195000001, 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19602166600000001, 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407771+0000", 2024-09-17T17:30:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1100000000000004e-07 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407775+0000", 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5510000000000001e-06 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603542+0000", 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19576712299999999 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603557+0000", 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4229999999999999e-05 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603658+0000", 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010186100000000001 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603791+0000", 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013299000000000001 2024-09-17T17:30:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.as3 62/32 ECSubWrite(tid=104, reqid=client.4566.0:558, at_version=62'84, trim_to=0'0, roll_forward_to=62'79))", 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 33.147982161000002, 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95386532199999996, 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501936+0000", 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0109999999999999e-06 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501948+0000", 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2544000000000001e-05 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454818+0000", 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95286919199999998 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454827+0000", 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3130000000000004e-06 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454959+0000", 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013207099999999999 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454969+0000", 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0368e-05 2024-09-17T17:30:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455761+0000", 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000792051 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455800+0000", 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8772000000000001e-05 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=123, reqid=client.4566.0:616, at_version=0'0, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.200104605, 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015260055, 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449813+0000", 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449823+0000", 2024-09-17T17:30:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9539999999999999e-06 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464497+0000", 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014673699 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464503+0000", 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1770000000000003e-06 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464557+0000", 2024-09-17T17:30:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464568+0000", 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0386e-05 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465032+0000", 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046443199999999997 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465072+0000", 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0280000000000001e-05 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=125, reqid=client.4566.0:619, at_version=68'49, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.183581182000001, 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010346985, 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466642+0000", 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030544600000000002 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475441+0000", 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087995179999999992 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475448+0000", 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4439999999999996e-06 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475494+0000", 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000003e-05 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475500+0000", 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476652+0000", 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001152887 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476683+0000", 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0306000000000001e-05 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.173845082, 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86543154899999997, 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476073+0000", 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4900000000000005e-07 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476093+0000", 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0431000000000001e-05 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340542+0000", 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86444875899999996 2024-09-17T17:30:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340554+0000", 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2017e-05 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340635+0000", 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1500000000000002e-05 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340651+0000", 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5444999999999999e-05 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341467+0000", 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00081634899999999996 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341503+0000", 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6298999999999999e-05 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:644 3.3s0 3:c77cc1f5:::smithi10238019-3:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.950823442000001, 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.524887887, 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 644 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699095+0000", 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.099e-06 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699100+0000", 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1549999999999998e-06 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699109+0000", 2024-09-17T17:30:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1670000000000005e-06 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699117+0000", 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8720000000000002e-06 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699171+0000", 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4237999999999998e-05 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699278+0000", 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107339 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699941+0000", 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066276400000000004 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.702383+0000", 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024413550000000001 2024-09-17T17:30:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223916+0000", 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52153379200000005 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223981+0000", 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5105999999999996e-05 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 69/67 ECSubWrite(tid=129, reqid=client.4566.0:621, at_version=0'0, trim_to=0'0, roll_forward_to=68'51))", 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.307129071999999, 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014148114, 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342789+0000", 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.13e-06 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342803+0000", 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4309e-05 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356208+0000", 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013404337000000001 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356219+0000", 2024-09-17T17:30:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1569000000000001e-05 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356293+0000", 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3894999999999996e-05 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356301+0000", 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5159999999999997e-06 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356892+0000", 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059179499999999999 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356936+0000", 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3563000000000003e-05 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:725 3.3s0 3:c5fa5218:::smithi10238019-30:head [create,setxattr n\\kYkdan\\thNN\\sTap[lWfmsQQctdU^aYUdXcPaN (278) in=40b,setxattr gdposbmqnPajgmVYW]VdX\\`gUQkaeU]UtbYUfiNg (1750) in=40b,setxattr ciVa^RQRQUnlPcOS[RhXiP[f`fVknjrV_`VY^mUX (417) in=40b,setxattr d_hhtYW (11296) in=7b,setxattr sRYR``_hO_QkVYhhONZZjoiUtaoqebe^l]rOq[_Y (1682) in=40b,setxattr _QNRabNbNqfbhqjYjpV`OpcjWYck]PSYrhXT_pR` (350) in=40b,setxattr Orajpk`Qfg^nQasaZW_ePrgt^]N__bT[NVS]PaaW (19224) in=40b,setxattr e[aOkpoq_NTqbYdOapYpT (11296) in=21b,setxattr ogiNjhNa]OiT]hhXbR_ti`jYmsgbs_rtq_jR`[`g (979) in=40b,setxattr j (11296) in=1b,setxattr N\\i[nSgaepe]]Wrqk[^`enQo]SlraVR^o_SO (11296) in=36b,setxattr cUpO`i`Te\\O\\Wtmgsdpghq]jam\\WhaNdZTYgepZZ (1049) in=40b,setxattr tU[pQ\\_U_siq^t[VT_bha\\Qjp`Y_n``SZlnPr`Nj (1329) in=40b,setxattr trgNslOknYrnhc`^OllRjkWjZc (11296) in=26b,setxattr ofS\\dVZbbra]Vf^gm[[Y^^XW`gWa_PjOdRlafjTa (66) in=40b,setxattr k^Ue`eUT^[jiTlg\\aPXs`ZXXaejRaY^X^\\oQ_clj (1262) in=40b,setxattr _goNc]^eaWViaTbdiVPjTWXpe_VrQgcY]nSajfQk (242) in=40b,setxattr sfdRTqXeR]_gb]lnNOUWXWfc\\NQcbOaTrTk`aOQq (700) in=40b,setxattr Yrh^hjspm\\YXTjTb]TRjpnsQU]sq\\c`RcfSeOchd (7436) in=40b,setxattr aahttaRSQ\\X`]orTWZRe[_OplTjd`YRoTQdeTmmj (1926) in=40b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.312252923999999, 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0205677150000001, 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 725 2024-09-17T17:30:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337666+0000", 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.513e-06 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337795+0000", 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000128961 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337808+0000", 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3713000000000001e-05 2024-09-17T17:30:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337819+0000", 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.027e-05 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341249+0000", 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034306190000000002 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:30:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341253+0000", 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4410000000000002e-06 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.346358+0000", 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.005105844 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.346550+0000", 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019203699999999999 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.347606+0000", 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001055896 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.353654+0000", 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060472520000000004 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358123+0000", 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044693620000000002 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358232+0000", 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108807 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 75/74 ECSubWriteReply(tid=94, last_complete=75'58, committed=1, applied=1))", 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.294773176, 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.74451308800000005, 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355148+0000", 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-06 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098760+0000", 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.74361193699999995 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098773+0000", 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.288e-05 2024-09-17T17:30:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098921+0000", 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014810999999999999 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.099657+0000", 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00073597300000000001 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:784 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~720896] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.298444641, 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010356648, 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 784 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351474+0000", 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.562e-06 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352295+0000", 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00082097799999999999 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352310+0000", 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5599000000000001e-05 2024-09-17T17:30:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352317+0000", 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0900000000000001e-06 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352366+0000", 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9768999999999997e-05 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352456+0000", 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0044999999999996e-05 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353483+0000", 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001026788 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357386+0000", 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039030559999999998 2024-09-17T17:30:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361725+0000", 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043392559999999997 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361829+0000", 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010350500000000001 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:785 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~622592] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.294875013999999, 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010812743, 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 785 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355044+0000", 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1919999999999999e-06 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355710+0000", 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066595000000000001 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355726+0000", 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6030999999999999e-05 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355735+0000", 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7390000000000005e-06 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355799+0000", 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4281000000000003e-05 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355877+0000", 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7553999999999996e-05 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.356754+0000", 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087738099999999999 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361867+0000", 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0051130380000000003 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365763+0000", 2024-09-17T17:30:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003895496 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365855+0000", 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2081000000000001e-05 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:786 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~688128] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.359228+0000", 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.290688824, 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0082478800000000008, 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 786 2024-09-17T17:30:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359228+0000", 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359228+0000", 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359230+0000", 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6360000000000001e-06 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359873+0000", 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064344499999999998 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359887+0000", 2024-09-17T17:30:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.397e-05 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359893+0000", 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6690000000000003e-06 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.359941+0000", 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8276999999999999e-05 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360014+0000", 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3243000000000003e-05 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360828+0000", 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00081379800000000002 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365404+0000", 2024-09-17T17:30:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045759479999999998 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367388+0000", 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001983552 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367476+0000", 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8342000000000002e-05 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:787 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~245760] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360589+0000", 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.289328029, 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010111284, 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 787 2024-09-17T17:30:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360589+0000", 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360589+0000", 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360590+0000", 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9699999999999995e-07 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360883+0000", 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029286399999999999 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360897+0000", 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4243000000000001e-05 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360906+0000", 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7909999999999993e-06 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361905+0000", 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00099930500000000003 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362013+0000", 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108478 2024-09-17T17:30:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362907+0000", 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00089352599999999996 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366879+0000", 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003972181 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.370649+0000", 2024-09-17T17:30:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003770022 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.370700+0000", 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1076999999999999e-05 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:788 3.13s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (57) in=7b] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.288998428999999, 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014979750999999999, 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 788 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4000000000000002e-07 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360925+0000", 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4010000000000002e-06 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360932+0000", 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1879999999999999e-06 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360937+0000", 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4330000000000003e-06 2024-09-17T17:30:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.364485+0000", 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035482949999999999 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.364590+0000", 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104424 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365073+0000", 2024-09-17T17:30:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048325000000000001 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367135+0000", 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002061677 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375816+0000", 2024-09-17T17:30:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086812480000000008 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375898+0000", 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2294999999999997e-05 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:789 3.13s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.288971618000001, 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016800492, 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 789 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360946+0000", 2024-09-17T17:30:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4300000000000003e-07 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360950+0000", 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.25e-06 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360958+0000", 2024-09-17T17:30:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5360000000000004e-06 2024-09-17T17:30:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360963+0000", 2024-09-17T17:30:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5509999999999999e-06 2024-09-17T17:30:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366330+0000", 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053669410000000001 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366434+0000", 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103384 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.376442+0000", 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010008349 2024-09-17T17:30:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.376609+0000", 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000166732 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377746+0000", 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001137206 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.12s5 85/84 ECSubRead(tid=64, to_read={3:4c833192:::smithi10238019-39:head=0,262144,0}, subchunks={3:4c833192:::smithi10238019-39:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.269345656, 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.043000999999999998, 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:30:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380573+0000", 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.576e-06 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380584+0000", 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1195e-05 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422791+0000", 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.042206670000000002 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422808+0000", 2024-09-17T17:30:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7677e-05 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422941+0000", 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013300900000000001 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.423572+0000", 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000630873 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s3 92/74 ECSubWrite(tid=317, reqid=osd.13.0:317, at_version=92'36, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.358767+0000", 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.2911499879999999, 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095933110000000002, 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358767+0000", 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358767+0000", 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358768+0000", 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7800000000000001e-07 2024-09-17T17:30:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358799+0000", 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1681999999999999e-05 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358836+0000", 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7085999999999997e-05 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358845+0000", 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8380000000000008e-06 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358922+0000", 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6648000000000002e-05 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358929+0000", 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6450000000000005e-06 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.368314+0000", 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093842620000000009 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.368360+0000", 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6572000000000001e-05 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:46.763 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:46.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:46.765 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:46.762+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:46.765 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:46.765 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:46.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:46.681+0000 7f86d3fff640 0 -- 172.21.15.7:0/4168684444 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f86d4221540 msgr2=0x7f86d4223930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:46.767 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:46.767 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:46.865 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:46.863+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:46.874 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:46.875 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:46.875 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:46.875 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:46.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:46.966 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:46.964+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:46.990 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:46.990 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:46.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:47.067 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:47.065+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:47.106 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:47.106 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:47.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:47.106 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:47.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:308 3.2s0 3:4c833192:::smithi10238019-39:head [rollback 5] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.133239435, 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076693619999999999, 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 308 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.68e-07 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970369+0000", 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7160000000000002e-06 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970378+0000", 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5739999999999996e-06 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970383+0000", 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0010000000000001e-06 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971568+0000", 2024-09-17T17:30:47.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011857020000000001 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971627+0000", 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8826999999999998e-05 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972103+0000", 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047574100000000001 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973681+0000", 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001578404 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.977986+0000", 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043046630000000002 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978033+0000", 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7166000000000002e-05 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.112 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:309 3.2s0 3:4c833192:::smithi10238019-39:head [append 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.133208392999997, 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077796990000000002, 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.113 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 309 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:47.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2e-07 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970399+0000", 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0779999999999997e-06 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970408+0000", 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7979999999999995e-06 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970415+0000", 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9310000000000002e-06 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972569+0000", 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021534969999999999 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972677+0000", 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108145 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973250+0000", 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057328100000000003 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973923+0000", 2024-09-17T17:30:47.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00067244899999999999 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978120+0000", 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041977949999999998 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978175+0000", 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4205e-05 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:327 3.2s0 3:497089da:::smithi10238019-9:head [rollback 5] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.404753972000002, 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0072194929999999996, 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 327 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698850+0000", 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698852+0000", 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3939999999999999e-06 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698858+0000", 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1480000000000003e-06 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698863+0000", 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3470000000000001e-06 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698988+0000", 2024-09-17T17:30:47.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012491999999999999 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698997+0000", 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6819999999999998e-06 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702506+0000", 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035088879999999999 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702618+0000", 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111428 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703259+0000", 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064172000000000003 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704549+0000", 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001289599 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706021+0000", 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001471857 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706069+0000", 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8072000000000003e-05 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:328 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.404714429999999, 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0073597310000000004, 2024-09-17T17:30:47.126 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.127 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.127 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 328 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:47.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7e-07 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698894+0000", 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.057e-06 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698902+0000", 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3760000000000002e-06 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698907+0000", 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0329999999999999e-06 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699035+0000", 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012849700000000001 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699042+0000", 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.55e-06 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703631+0000", 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004589536 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703735+0000", 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103891 2024-09-17T17:30:47.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704152+0000", 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041683699999999998 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704821+0000", 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066855199999999999 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706206+0000", 2024-09-17T17:30:47.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001385788 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706249+0000", 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2243999999999997e-05 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 52/35 ECSubWrite(tid=125, reqid=osd.13.0:125, at_version=52'57, trim_to=0'0, roll_forward_to=51'53))", 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 43.759522466999996, 2024-09-17T17:30:47.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0071485719999999997, 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344081+0000", 2024-09-17T17:30:47.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344082+0000", 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.85e-07 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344108+0000", 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6270000000000001e-05 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344147+0000", 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9462999999999998e-05 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344158+0000", 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0786e-05 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344212+0000", 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4143e-05 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.344220+0000", 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8390000000000007e-06 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.351178+0000", 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006957876 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.351229+0000", 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1409999999999997e-05 2024-09-17T17:30:47.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:421 3.2s0 3:4c833192:::smithi10238019-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 43.720962206000003, 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090389679999999997, 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 421 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382642+0000", 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000005e-07 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382644+0000", 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382652+0000", 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5059999999999998e-06 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382656+0000", 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8029999999999996e-06 2024-09-17T17:30:47.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385875+0000", 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0032189900000000001 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385954+0000", 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9087999999999999e-05 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388823+0000", 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028687090000000001 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388909+0000", 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5457000000000005e-05 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.391680+0000", 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027713939999999999 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.143 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 53/35 ECSubWrite(tid=133, reqid=osd.13.0:133, at_version=53'64, trim_to=0'0, roll_forward_to=52'62))", 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 42.741559021, 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078716470000000007, 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362068+0000", 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3370000000000002e-05 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362467+0000", 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039876899999999999 2024-09-17T17:30:47.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362476+0000", 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8740000000000001e-06 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364044+0000", 2024-09-17T17:30:47.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001568261 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364055+0000", 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.077e-05 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369861+0000", 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0058066660000000003 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369916+0000", 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4691999999999997e-05 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s5 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 39.764185882, 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074908880000000002, 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339418+0000", 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339449+0000", 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.133e-05 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339554+0000", 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104682 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339563+0000", 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9129999999999992e-06 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339937+0000", 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037396800000000002 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339947+0000", 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0451999999999999e-05 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346872+0000", 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069250020000000004 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346908+0000", 2024-09-17T17:30:47.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6117e-05 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=154, reqid=osd.13.0:154, at_version=57'84, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 38.713688410000003, 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035993311, 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.154 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389916+0000", 2024-09-17T17:30:47.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0299999999999997e-07 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389936+0000", 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0061e-05 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425016+0000", 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035080622999999998 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425025+0000", 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3110000000000006e-06 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425088+0000", 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3442999999999995e-05 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425103+0000", 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4625e-05 2024-09-17T17:30:47.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425875+0000", 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000772585 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425908+0000", 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2759999999999998e-05 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=155, reqid=osd.13.0:155, at_version=57'86, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 38.712787231, 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26589276499999998, 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390817+0000", 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390839+0000", 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2725999999999999e-05 2024-09-17T17:30:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655507+0000", 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26466748600000001 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655515+0000", 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5530000000000004e-06 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655625+0000", 2024-09-17T17:30:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109997 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655635+0000", 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0000000000000001e-05 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656673+0000", 2024-09-17T17:30:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001038464 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656709+0000", 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5830000000000001e-05 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=156, reqid=client.4566.0:492, at_version=57'10, trim_to=0'0, roll_forward_to=43'8))", 2024-09-17T17:30:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 38.678626756, 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23215693400000001, 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4900000000000004e-07 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424998+0000", 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0800000000000001e-05 2024-09-17T17:30:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655688+0000", 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23068955499999999 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655696+0000", 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0839999999999992e-06 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655818+0000", 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012232399999999999 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655830+0000", 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1355000000000001e-05 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657105+0000", 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001275403 2024-09-17T17:30:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657133+0000", 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8464000000000001e-05 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=158, reqid=client.4566.0:492, at_version=0'0, trim_to=0'0, roll_forward_to=57'10))", 2024-09-17T17:30:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 38.445781693000001, 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26690073399999997, 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000001e-07 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657834+0000", 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1450999999999999e-05 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923785+0000", 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26595101999999998 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923800+0000", 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.508e-05 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923956+0000", 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155876 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923975+0000", 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9443e-05 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924688+0000", 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071237099999999995 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924722+0000", 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4629000000000003e-05 2024-09-17T17:30:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 59/35 ECSubWrite(tid=164, reqid=client.4566.0:513, at_version=59'90, trim_to=0'0, roll_forward_to=59'88))", 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.728061015000002, 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080861479999999996, 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375543+0000", 2024-09-17T17:30:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4499999999999995e-07 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375918+0000", 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037515399999999999 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375961+0000", 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2206000000000002e-05 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375969+0000", 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8349999999999993e-06 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376050+0000", 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0269e-05 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376060+0000", 2024-09-17T17:30:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0509999999999999e-05 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383584+0000", 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075235939999999998 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383628+0000", 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4635000000000002e-05 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:562 3.2s0 3:497089da:::smithi10238019-9:head [rollback f] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 32.687469684, 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076874359999999997, 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 562 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1299999999999999e-07 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416138+0000", 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4920000000000002e-06 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416146+0000", 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8800000000000008e-06 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416150+0000", 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6840000000000004e-06 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416271+0000", 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012026799999999999 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416278+0000", 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5889999999999997e-06 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419622+0000", 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0033434419999999999 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419740+0000", 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011798600000000001 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420292+0000", 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00055223900000000005 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421629+0000", 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001337429 2024-09-17T17:30:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423776+0000", 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021470370000000001 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423821+0000", 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4676999999999999e-05 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:563 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 32.687445832000002, 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079319400000000002, 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 563 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416158+0000", 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416161+0000", 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6649999999999999e-06 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416171+0000", 2024-09-17T17:30:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0176e-05 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416181+0000", 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0327e-05 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416310+0000", 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129247 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416317+0000", 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0720000000000002e-06 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420645+0000", 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043279629999999998 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420731+0000", 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5937000000000001e-05 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421175+0000", 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044333000000000003 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423834+0000", 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002659121 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424046+0000", 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021196999999999999 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424089+0000", 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3702999999999998e-05 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:907 3.12s0 3:497089da:::smithi10238019-9:head [copy-from ver 0] snapc 1f=[1f,1e,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.372716+0000", 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7308876919999996, 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079506950000000007, 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 907 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372716+0000", 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372716+0000", 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372718+0000", 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372740+0000", 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2580000000000001e-05 2024-09-17T17:30:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372766+0000", 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5984000000000001e-05 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372784+0000", 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7949e-05 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372887+0000", 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102662 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373071+0000", 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018410400000000001 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380666+0000", 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075953660000000001 2024-09-17T17:30:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:908 3.12s0 3:497089da:::smithi10238019-9:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.372809+0000", 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7307943559999996, 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.008374968, 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 908 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372809+0000", 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372809+0000", 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372810+0000", 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372813+0000", 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8199999999999998e-06 2024-09-17T17:30:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373013+0000", 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019992100000000001 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373022+0000", 2024-09-17T17:30:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8089999999999999e-06 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373804+0000", 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00078201900000000001 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373866+0000", 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1671e-05 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380885+0000", 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0070188480000000003 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.381046+0000", 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016086799999999999 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.381184+0000", 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013826600000000001 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s5 92/75 ECSubWrite(tid=212, reqid=osd.12.0:212, at_version=92'80, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.749453382, 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0089937200000000002, 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7899999999999999e-07 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354174+0000", 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3799999999999999e-05 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354419+0000", 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024507199999999999 2024-09-17T17:30:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354438+0000", 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8819000000000001e-05 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354944+0000", 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050620199999999998 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354954+0000", 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.009e-05 2024-09-17T17:30:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363090+0000", 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081354419999999997 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363144+0000", 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3915999999999997e-05 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s2 92/74 ECSubWrite(tid=316, reqid=osd.13.0:316, at_version=92'34, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.356287+0000", 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.747316305, 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012103668, 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356287+0000", 2024-09-17T17:30:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356287+0000", 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356288+0000", 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8899999999999998e-07 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356316+0000", 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7753e-05 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358713+0000", 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023969759999999999 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358722+0000", 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9139999999999997e-06 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358826+0000", 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010438 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358838+0000", 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1817000000000001e-05 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.368219+0000", 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093809289999999997 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.368391+0000", 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017191 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s2 92/74 ECSubWrite(tid=317, reqid=osd.13.0:317, at_version=92'36, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7447717680000001, 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.096070654000000005, 2024-09-17T17:30:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:30:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1100000000000002e-07 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358851+0000", 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8972999999999999e-05 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453479+0000", 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.094628491999999995 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453495+0000", 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498999999999998e-05 2024-09-17T17:30:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453664+0000", 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016951999999999999 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453697+0000", 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2301999999999997e-05 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.454861+0000", 2024-09-17T17:30:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001164099 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.454902+0000", 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1458000000000002e-05 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:47.217 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:47.219 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:47.220 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:47.217+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:47.320 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:47.318+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:47.337 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:47.337 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:47.337 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:47.337 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:47.345 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:30:47.347 INFO:tasks.rados.rados.0.smithi102.stdout:366: done (1 left) 2024-09-17T17:30:47.348 INFO:tasks.rados.rados.0.smithi102.stdout:367: read oid 13 snap 19 2024-09-17T17:30:47.348 INFO:tasks.rados.rados.0.smithi102.stdout:367: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:47.348 INFO:tasks.rados.rados.0.smithi102.stdout:368: copy_from oid 19 from oid 45 current snap is 31 2024-09-17T17:30:47.348 INFO:tasks.rados.rados.0.smithi102.stdout:365: done (2 left) 2024-09-17T17:30:47.348 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 41 current snap is 31 2024-09-17T17:30:47.348 INFO:tasks.rados.rados.0.smithi102.stdout:369: seq_num 156 ranges {2015232=229376} 2024-09-17T17:30:47.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:47.359 INFO:tasks.rados.rados.0.smithi102.stdout:369: writing smithi10238019-41 from 2015232 to 2244608 tid 1 2024-09-17T17:30:47.359 INFO:tasks.rados.rados.0.smithi102.stdout:367: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-17T17:30:47.359 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:47.362 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:47.365 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:47.368 INFO:tasks.rados.rados.0.smithi102.stdout:367: done (2 left) 2024-09-17T17:30:47.368 INFO:tasks.rados.rados.0.smithi102.stdout:370: delete oid 1 current snap is 31 2024-09-17T17:30:47.370 INFO:tasks.rados.rados.0.smithi102.stdout:369: finishing write tid 1 to smithi10238019-41 2024-09-17T17:30:47.370 INFO:tasks.rados.rados.0.smithi102.stdout:369: finishing write tid 2 to smithi10238019-41 2024-09-17T17:30:47.371 INFO:tasks.rados.rados.0.smithi102.stdout:369: finishing write tid 3 to smithi10238019-41 2024-09-17T17:30:47.371 INFO:tasks.rados.rados.0.smithi102.stdout:369: oid 41 updating version 0 to 47 2024-09-17T17:30:47.371 INFO:tasks.rados.rados.0.smithi102.stdout:369: oid 41 version 47 is already newer than 46 2024-09-17T17:30:47.371 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 41 v 47 (ObjNum 156 snap 31 seq_num 156) dirty exists 2024-09-17T17:30:47.371 INFO:tasks.rados.rados.0.smithi102.stdout:369: left oid 41 (ObjNum 156 snap 31 seq_num 156) 2024-09-17T17:30:47.377 INFO:tasks.rados.rados.0.smithi102.stdout:369: done (2 left) 2024-09-17T17:30:47.378 INFO:tasks.rados.rados.0.smithi102.stdout:370: done (1 left) 2024-09-17T17:30:47.378 INFO:tasks.rados.rados.0.smithi102.stdout:371: snap_create 2024-09-17T17:30:47.383 INFO:tasks.rados.rados.0.smithi102.stdout:368: finishing copy_from racing read to smithi10238019-19 2024-09-17T17:30:47.421 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:47.419+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:47.436 INFO:tasks.rados.rados.0.smithi102.stdout:368: finishing copy_from to smithi10238019-19 2024-09-17T17:30:47.436 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 19 v 153 (ObjNum 142 snap 27 seq_num 142) dirty exists 2024-09-17T17:30:47.466 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:47.466 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:47.466 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:47.466 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:47.466 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:47.474 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:47.522 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:47.520+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:47.620 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:169 3.as0 3:5ee91470:::smithi10238019-6:head [read 83682~14622,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 58.626687717000003, 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67448544799999999, 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 169 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990278+0000", 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6400000000000002e-07 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.623 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990283+0000", 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3900000000000001e-06 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990291+0000", 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9379999999999992e-06 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990297+0000", 2024-09-17T17:30:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6099999999999997e-06 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992308+0000", 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0020111650000000001 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992380+0000", 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1459999999999997e-05 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.664763+0000", 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67238332099999998 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 37/32 ECSubWriteReply(tid=24, last_complete=37'12, committed=1, applied=1))", 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 58.616655874999999, 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069166769000000003, 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:47.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000310+0000", 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1299999999999998e-07 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000314+0000", 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.461e-06 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069373+0000", 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069059000999999995 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069380+0000", 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2570000000000001e-06 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069437+0000", 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7519999999999998e-05 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069476+0000", 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8617e-05 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.630 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 58.547621559, 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52672497699999998, 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069345+0000", 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.048e-06 2024-09-17T17:30:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069355+0000", 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0052e-05 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595186+0000", 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52583139400000001 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595201+0000", 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4790000000000001e-05 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595348+0000", 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014743699999999999 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595363+0000", 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4589999999999999e-05 2024-09-17T17:30:47.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596033+0000", 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067039499999999995 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596069+0000", 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5271000000000002e-05 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:223 3.as0 3:5a070e56:::smithi10238019-47:head [read 351913~434519,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 56.890952267999999, 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010342224000000001, 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.636 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 223 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726013+0000", 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7299999999999996e-07 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726018+0000", 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0770000000000001e-06 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726027+0000", 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7210000000000006e-06 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.726034+0000", 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3419999999999996e-06 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.728268+0000", 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002233886 2024-09-17T17:30:47.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.728356+0000", 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8360000000000001e-05 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.736355+0000", 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079992650000000002 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.640 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:390 3.as0 3:5ee91470:::smithi10238019-6:head [create,setxattr `sjUtrS\\bejkdS^PZ^UbhfeR]sl^p__anSUR_aom (1578) in=40b,setxattr Xgg (981) in=3b,setxattr jjU_`S[ao\\\\^aUS`bZo^cefhdeaegSZfc]sS[m[k (631) in=40b,setxattr WptVXs^]ZljihqWVjRTTRsYbpoOd[kede^[d^iNS (15648) in=40b,setxattr NUPa_VdcRWiTY (981) in=13b,setxattr oapqrhbaXndiek^ec[NZPnlNjeX^`^^ht`__ll[g (630) in=40b,setxattr hUokalb_dNqZaP[SZdSptRb]ZndqQTkPSdWpXlg_ (1683) in=40b,setxattr SOaUaOtoOmejUfsqfadVc_\\fpWak\\^nknRQfaXOZ (9120) in=40b,setxattr r[RO_Pl (981) in=7b,setxattr OOd^ZXYi[]NO^mlSkQafkPlVaY`UYcd\\bnaQUZjq (1225) in=40b,setxattr mlY]_qqgYNhaYetZtkgQPN^a^VmW[amS`Qbadgjr (137) in=40b,setxattr iVYdjchT[j\\[\\\\Uto`qnjUcXas^PtqZa^VoRktTT (1823) in=40b,setxattr PhV`]ZkggalnUas (981) in=15b,setxattr [TNbUtYOkT`ZatPerOQOlk\\WahmgadNWaeRZ^YT\\ (1121) in=40b,setxattr aRqq]f]b_cobaZ`ZZRaattjja[X``PaT\\l\\^Y[jU (313) in=40b,setxattr qYXjVRWg (981) in=8b,setxattr cm`nfp^lmRcSZUsStaOg_omiONqeTRWUsOjm]png (1227) in=40b,setxattr _jtbmRhQeZe]nY\\TQctPVagWm[isibp]ooW`hq[Z (1367) in=40b,setxattr _Ybhal\\]RPkmpUPPRRb[_[YfQpj[aXsgkfNbeeOZ (946) in=40b,setxattr YeQo_aSrmVSirmpTkkZqfZaaqOtg\\X\\qoRPrjPai (1786) in=40b] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.284549046999999, 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096219809999999999, 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.641 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 390 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332416+0000", 2024-09-17T17:30:47.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332418+0000", 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3379999999999999e-06 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332463+0000", 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4666999999999998e-05 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332482+0000", 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8822e-05 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332493+0000", 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0686000000000001e-05 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332566+0000", 2024-09-17T17:30:47.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3250999999999994e-05 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.332571+0000", 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1479999999999997e-06 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.335092+0000", 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0025209799999999999 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.335166+0000", 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3672999999999997e-05 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.336267+0000", 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011013399999999999 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.338033+0000", 2024-09-17T17:30:47.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001766323 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.341931+0000", 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0038977349999999998 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.342038+0000", 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107018 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.647 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:396 3.as0 3:5280ab18:::smithi10238019-18:head [append 0~147456] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.273775702000002, 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012450536999999999, 2024-09-17T17:30:47.648 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 396 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343189+0000", 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343191+0000", 2024-09-17T17:30:47.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0839999999999999e-06 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343250+0000", 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9113999999999997e-05 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343262+0000", 2024-09-17T17:30:47.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2649999999999999e-05 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343270+0000", 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0879999999999995e-06 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.345733+0000", 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024625039999999999 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.345807+0000", 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3769000000000005e-05 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346281+0000", 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047393899999999999 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.350852+0000", 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0045715920000000002 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355563+0000", 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047105319999999999 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355640+0000", 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.7265e-05 2024-09-17T17:30:47.654 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:397 3.as0 3:5280ab18:::smithi10238019-18:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.273675425, 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012559297000000001, 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.655 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 397 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343290+0000", 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3399999999999999e-07 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343293+0000", 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1700000000000001e-06 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343299+0000", 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3199999999999999e-06 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343302+0000", 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.495e-06 2024-09-17T17:30:47.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346935+0000", 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0036331419999999998 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.346988+0000", 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2828999999999999e-05 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347263+0000", 2024-09-17T17:30:47.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000274999 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.350964+0000", 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003701101 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355764+0000", 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047997869999999998 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.355849+0000", 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4919999999999993e-05 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:398 3.as0 3:5280ab18:::smithi10238019-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.273657338, 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013718741, 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.661 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 398 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9700000000000003e-07 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343310+0000", 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.525e-06 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343314+0000", 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8770000000000003e-06 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343316+0000", 2024-09-17T17:30:47.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8080000000000001e-06 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347523+0000", 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042062560000000002 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347583+0000", 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0201999999999997e-05 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356234+0000", 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086509659999999995 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356431+0000", 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00019758699999999999 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.357027+0000", 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00059522299999999998 2024-09-17T17:30:47.666 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 51/32 ECSubWriteReply(tid=64, last_complete=51'43, committed=1, applied=1))", 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.260190829999999, 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.78010271600000003, 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356775+0000", 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5000000000000001e-07 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356781+0000", 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.716e-06 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136139+0000", 2024-09-17T17:30:47.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.77935822499999996 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136153+0000", 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.359e-05 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136325+0000", 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017257700000000001 2024-09-17T17:30:47.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136877+0000", 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00055165800000000001 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s2 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:47.671 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 40.279036339999998, 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012584702999999999, 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337929+0000", 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.337956+0000", 2024-09-17T17:30:47.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6574e-05 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338017+0000", 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1050999999999995e-05 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338024+0000", 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2300000000000002e-06 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338194+0000", 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017029299999999999 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.338202+0000", 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.892e-06 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.350495+0000", 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012292684 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.350514+0000", 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8678999999999999e-05 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:641 3.as0 3:5280ab18:::smithi10238019-18:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 28.093267220000001, 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.16562932699999999, 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 641 2024-09-17T17:30:47.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523700+0000", 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1150000000000001e-06 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523708+0000", 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5599999999999996e-06 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523725+0000", 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7725999999999999e-05 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523733+0000", 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5299999999999999e-06 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6327000000000004e-05 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523914+0000", 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114837 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524640+0000", 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072579000000000003 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526382+0000", 2024-09-17T17:30:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0017423060000000001 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689252+0000", 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.162869823 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689327+0000", 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5313000000000002e-05 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.682 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:655 3.as0 3:5673cc6d:::smithi10238019-2:head [read 0~1818624,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347150+0000", 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 27.269814826000001, 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096743809999999993, 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.683 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 655 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347150+0000", 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347150+0000", 2024-09-17T17:30:47.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347151+0000", 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0900000000000002e-07 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347154+0000", 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0230000000000001e-06 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347161+0000", 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.046e-06 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347166+0000", 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9919999999999998e-06 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347342+0000", 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017651599999999999 2024-09-17T17:30:47.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347346+0000", 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1609999999999999e-06 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351287+0000", 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0039411480000000002 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351386+0000", 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9679000000000006e-05 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.356825+0000", 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0054383069999999999 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:661 3.as0 3:5a070e56:::smithi10238019-47:head [copy-from ver 88] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 27.269789177, 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.026523057999999999, 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 661 2024-09-17T17:30:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347177+0000", 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347181+0000", 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5050000000000001e-06 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347187+0000", 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1990000000000003e-06 2024-09-17T17:30:47.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347192+0000", 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2969999999999997e-06 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347359+0000", 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016719099999999999 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347361+0000", 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7039999999999999e-06 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351847+0000", 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.004485274 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351930+0000", 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2813999999999997e-05 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.367807+0000", 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015877189999999999 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.370285+0000", 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024778510000000001 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.373602+0000", 2024-09-17T17:30:47.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0033175460000000002 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.373699+0000", 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6977000000000003e-05 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:662 3.as0 3:5a070e56:::smithi10238019-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 27.269716897999999, 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.022338108999999998, 2024-09-17T17:30:47.695 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 662 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347249+0000", 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2099999999999997e-07 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347251+0000", 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6589999999999999e-06 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347258+0000", 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4330000000000001e-06 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347263+0000", 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3979999999999998e-06 2024-09-17T17:30:47.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347385+0000", 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000121845 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347390+0000", 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.147e-06 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.699 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352716+0000", 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053252270000000001 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352767+0000", 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1344999999999997e-05 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369400+0000", 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016633254 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369488+0000", 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7441999999999994e-05 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369586+0000", 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8838000000000005e-05 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.701 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:663 3.as0 3:5ee91470:::smithi10238019-6:head [delete] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e69)", 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 27.269694518000001, 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016232469999999999, 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.702 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 663 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:47.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1300000000000001e-07 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347273+0000", 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2800000000000002e-06 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347279+0000", 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2170000000000002e-06 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347283+0000", 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4719999999999997e-06 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347416+0000", 2024-09-17T17:30:47.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013269700000000001 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347421+0000", 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1019999999999996e-06 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352795+0000", 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053736970000000002 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352873+0000", 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8318000000000005e-05 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.353539+0000", 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00066570400000000004 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.356943+0000", 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034039679999999998 2024-09-17T17:30:47.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.363452+0000", 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.006509384 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.363503+0000", 2024-09-17T17:30:47.708 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1218000000000003e-05 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 73/32 ECSubWriteReply(tid=139, last_complete=73'108, committed=1, applied=1))", 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.874747386999999, 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.257005174, 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.709 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742222+0000", 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4359999999999998e-06 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998674+0000", 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.25645215799999999 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998683+0000", 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998749+0000", 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5689000000000001e-05 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.999223+0000", 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047410699999999998 2024-09-17T17:30:47.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:736 3.as0 3:5ceb0529:::smithi10238019-12:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.232980606000002, 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.95058956999999999, 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.713 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 736 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.44e-07 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383988+0000", 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.926e-06 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383994+0000", 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2729999999999999e-06 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384000+0000", 2024-09-17T17:30:47.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.237e-06 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385207+0000", 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001207079 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385300+0000", 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3450999999999997e-05 2024-09-17T17:30:47.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385740+0000", 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000440312 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386613+0000", 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00087278900000000001 2024-09-17T17:30:47.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334485+0000", 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94787173599999996 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334574+0000", 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9222999999999996e-05 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:737 3.as0 3:5ceb0529:::smithi10238019-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.232957707000001, 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.95178945199999998, 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 737 2024-09-17T17:30:47.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384008+0000", 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5299999999999999e-07 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384011+0000", 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.852e-06 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.722 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384018+0000", 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3860000000000001e-06 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384023+0000", 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7790000000000003e-06 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386679+0000", 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0026562550000000002 2024-09-17T17:30:47.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386739+0000", 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0207999999999997e-05 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334966+0000", 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94822698000000005 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.724 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.335064+0000", 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7929000000000004e-05 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.335797+0000", 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00073260999999999997 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.725 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.10.22:3 3.16s0 3:6d14b604:::smithi10238019-45:head [assert-version v153,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.359098+0000", 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.25786681, 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015040585, 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359098+0000", 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359098+0000", 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359100+0000", 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.57e-06 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.727 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359106+0000", 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6219999999999997e-06 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359119+0000", 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3735e-05 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359130+0000", 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.022e-05 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359299+0000", 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016968300000000001 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for peered", 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359303+0000", 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0369999999999996e-06 2024-09-17T17:30:47.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.365427+0000", 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0061239780000000004 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.365942+0000", 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00051429299999999998 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.374139+0000", 2024-09-17T17:30:47.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0081974470000000001 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 96/95 ECSubWriteReply(tid=175, last_complete=96'137, committed=1, applied=1))", 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.239151421, 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089945729000000002, 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:47.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:30:47.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4600000000000004e-07 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377819+0000", 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3170000000000003e-06 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467660+0000", 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089841455000000001 2024-09-17T17:30:47.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467686+0000", 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5420999999999999e-05 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467743+0000", 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7120000000000002e-05 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467759+0000", 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.677e-05 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:47.735 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:47.737 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:47.737 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:47.734+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:47.837 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:47.836+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:47.883 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:47.883 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:47.884 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:47.884 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:47.892 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:47.977 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:47.975+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:48.034 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:48.034 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:48.035 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:48.035 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:48.035 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:48.039 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:48.037+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:48.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:48.140 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:48.139+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:48.187 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:311 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 431326~1305378,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 53.214655831000002, 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085766909999999991, 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.188 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 311 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.189 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970413+0000", 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970414+0000", 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.8299999999999997e-07 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.190 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.191 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970418+0000", 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9500000000000003e-06 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970425+0000", 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.452e-06 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970429+0000", 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4240000000000002e-06 2024-09-17T17:30:48.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.971012+0000", 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00058375099999999997 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.971104+0000", 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1195999999999999e-05 2024-09-17T17:30:48.193 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978990+0000", 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078864350000000007 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:312 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 0~1736704,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 53.214627096000001, 2024-09-17T17:30:48.194 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50817852500000005, 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 312 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.195 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1600000000000002e-07 2024-09-17T17:30:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970445+0000", 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7999999999999999e-06 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970451+0000", 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0970000000000002e-06 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970456+0000", 2024-09-17T17:30:48.197 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6009999999999997e-06 2024-09-17T17:30:48.198 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.198 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976257+0000", 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005800889 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976348+0000", 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1614999999999996e-05 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.199 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478621+0000", 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50227210700000002 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.200 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.ds0 43/32 ECSubReadReply(tid=23, attrs_read=0))", 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 53.208016403999999, 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.501606106, 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.201 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977054+0000", 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.16e-07 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978014+0000", 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096069199999999997 2024-09-17T17:30:48.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477737+0000", 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.49972330100000001 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477745+0000", 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2130000000000002e-06 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.203 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477802+0000", 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7555e-05 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478659+0000", 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00085652899999999997 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=31, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 47.827198271999997, 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53920807500000001, 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.205 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:30:48.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357876+0000", 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.092e-06 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896536+0000", 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53865993499999998 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896547+0000", 2024-09-17T17:30:48.207 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1608e-05 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896653+0000", 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000105807 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.208 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897079+0000", 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042597099999999998 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=32, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:48.209 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 47.287567103000001, 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222760399, 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.210 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897503+0000", 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.68e-07 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.211 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897507+0000", 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7709999999999997e-06 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120082+0000", 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222574243 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.212 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120095+0000", 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.291e-05 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120199+0000", 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010402499999999999 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120262+0000", 2024-09-17T17:30:48.213 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3781999999999993e-05 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 51/32 ECSubWrite(tid=61, reqid=osd.3.0:61, at_version=51'39, trim_to=0'0, roll_forward_to=50'37))", 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.844947746000003, 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007939804, 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340121+0000", 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.215 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340122+0000", 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4200000000000005e-07 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340146+0000", 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3578e-05 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340199+0000", 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3127999999999997e-05 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340208+0000", 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3589999999999997e-06 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.217 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340279+0000", 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0772000000000006e-05 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.340289+0000", 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9860000000000006e-06 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.347977+0000", 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076882779999999998 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.218 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.348061+0000", 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3861000000000002e-05 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.219 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:412 3.ds0 3:badfad2a:::smithi10238019-37:head [append 0~655360] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.828352293000002, 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097729180000000002, 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 412 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.220 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.221 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356719+0000", 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.255e-06 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357255+0000", 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00053548099999999998 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357270+0000", 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5045000000000001e-05 2024-09-17T17:30:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357278+0000", 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8429999999999999e-06 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357346+0000", 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7186000000000004e-05 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357460+0000", 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114388 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.358484+0000", 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023641 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366119+0000", 2024-09-17T17:30:48.224 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076354270000000002 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366416+0000", 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00029701100000000002 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366490+0000", 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3640999999999995e-05 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.225 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:415 3.ds0 3:badfad2a:::smithi10238019-37:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.823158632000002, 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089535759999999995, 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 415 2024-09-17T17:30:48.226 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.227 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361911+0000", 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7e-07 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361913+0000", 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2350000000000002e-06 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.228 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361919+0000", 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4750000000000001e-06 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361922+0000", 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3110000000000001e-06 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.229 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366710+0000", 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047879840000000003 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366770+0000", 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9886000000000002e-05 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.230 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367045+0000", 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027536599999999998 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370540+0000", 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034951610000000001 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370583+0000", 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2682999999999998e-05 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.231 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370864+0000", 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00028100500000000002 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:416 3.ds0 3:badfad2a:::smithi10238019-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.823141606, 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096567449999999996, 2024-09-17T17:30:48.232 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 416 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.233 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361928+0000", 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5400000000000002e-07 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361929+0000", 2024-09-17T17:30:48.234 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.471e-06 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361934+0000", 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1100000000000002e-06 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361937+0000", 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1499999999999999e-06 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.235 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367591+0000", 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005653534 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367664+0000", 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2779000000000002e-05 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370895+0000", 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0032314409999999998 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.236 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.237 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370943+0000", 2024-09-17T17:30:48.237 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7886999999999997e-05 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.371584+0000", 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000641119 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.238 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 56/32 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=56'57, trim_to=0'0, roll_forward_to=53'55))", 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 40.841449433999998, 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089122520000000007, 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.239 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.92e-07 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343638+0000", 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8074e-05 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.241 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343672+0000", 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343680+0000", 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8979999999999996e-06 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343804+0000", 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012350499999999999 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.242 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343814+0000", 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0353000000000001e-05 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352491+0000", 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086771880000000006 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352532+0000", 2024-09-17T17:30:48.243 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0837999999999998e-05 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 74/32 ECSubWrite(tid=148, reqid=client.4566.0:736, at_version=74'114, trim_to=0'0, roll_forward_to=74'111))", 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.799353562, 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94858601300000001, 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.244 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.245 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5599999999999995e-07 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385733+0000", 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7152000000000001e-05 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.246 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333728+0000", 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94799460099999999 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333734+0000", 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6109999999999999e-06 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.247 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333806+0000", 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1377000000000005e-05 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333813+0000", 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2969999999999998e-06 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334272+0000", 2024-09-17T17:30:48.248 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000458956 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334302+0000", 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9462999999999999e-05 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.16s5 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:30:48.249 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.821221663999999, 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011160745999999999, 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:30:48.250 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363848+0000", 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7400000000000003e-07 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.251 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363866+0000", 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8411999999999999e-05 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363893+0000", 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6346000000000002e-05 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363899+0000", 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5220000000000004e-06 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.252 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363943+0000", 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3933000000000001e-05 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363948+0000", 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0810000000000001e-06 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.374983+0000", 2024-09-17T17:30:48.253 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011034284 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.375008+0000", 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5593999999999999e-05 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 78/32 ECSubWriteReply(tid=78, last_complete=78'67, committed=1, applied=1))", 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.540819542000001, 2024-09-17T17:30:48.254 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.370820394, 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4000000000000001e-07 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644254+0000", 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.924e-06 2024-09-17T17:30:48.256 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014178+0000", 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.369923422 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014192+0000", 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4722999999999999e-05 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.257 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014343+0000", 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015040400000000001 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.015070+0000", 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072728099999999998 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.258 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 78/32 ECSubWriteReply(tid=79, last_complete=78'67, committed=1, applied=1))", 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.168974118000001, 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.47947288999999998, 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.259 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016096+0000", 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7000000000000005e-07 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.260 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016099+0000", 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2660000000000002e-06 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495222+0000", 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.47912339300000001 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.261 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495235+0000", 2024-09-17T17:30:48.262 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3134e-05 2024-09-17T17:30:48.262 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.262 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.262 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.262 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495445+0000", 2024-09-17T17:30:48.262 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000209302 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495568+0000", 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000123225 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.263 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 85/84 [PushOp(3:497089da:::smithi10238019-9:13, version: 71'47, data_included: [0~36864], data_size: 36864, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:13@71'47, size: 147456, copy_subset: [], clone_subset: {}, snapset: 1b={13: [13, 10], 1b: [1b, 1a, 18, 17, 16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 147456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.744977677, 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.43750746400000001, 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:30:48.264 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440092+0000", 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8999999999999996e-07 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.265 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440171+0000", 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9203999999999996e-05 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877162+0000", 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.436990934 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.266 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877172+0000", 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0164e-05 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877272+0000", 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9265000000000006e-05 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877599+0000", 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00032730699999999999 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.267 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 85/84 [PushOp(3:497089da:::smithi10238019-9:1b, version: 82'53, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:1b@82'53, size: 819200, copy_subset: [], clone_subset: {}, snapset: 1b={13: [13, 10], 1b: [1b, 1a, 18, 17, 16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 819200, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 11.295351728, 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.46291758199999999, 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:30:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889718+0000", 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8200000000000008e-07 2024-09-17T17:30:48.269 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889996+0000", 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027726200000000002 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351714+0000", 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.46171880599999998 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351746+0000", 2024-09-17T17:30:48.270 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1831000000000001e-05 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351912+0000", 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016581899999999999 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.271 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.352635+0000", 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000722882 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 86/84 [PushOp(3:497089da:::smithi10238019-9:head, version: 82'58, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:head@82'58, size: 3112960, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 10.833248197, 2024-09-17T17:30:48.272 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.129055472, 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.273 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.274 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.275 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:30:48.275 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8800000000000002e-07 2024-09-17T17:30:48.275 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.275 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.275 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.275 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.352494+0000", 2024-09-17T17:30:48.276 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067254700000000001 2024-09-17T17:30:48.276 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.276 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.276 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.276 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480390+0000", 2024-09-17T17:30:48.276 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.12789568900000001 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480399+0000", 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7610000000000002e-06 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.277 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480484+0000", 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4571000000000001e-05 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480876+0000", 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039231599999999998 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.278 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:885 3.ds0 3:bc68f0c8:::smithi10238019-15:head [rollback 1f] snapc 1f=[1f,1e,1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:41.340293+0000", 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.8447763369999999, 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0087990250000000002, 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 885 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.279 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340293+0000", 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340293+0000", 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340293+0000", 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3700000000000001e-07 2024-09-17T17:30:48.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340295+0000", 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1660000000000001e-06 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340302+0000", 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7850000000000003e-06 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340307+0000", 2024-09-17T17:30:48.281 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.011e-06 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340414+0000", 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000106977 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340427+0000", 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3125000000000001e-05 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.282 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.344189+0000", 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0037613220000000001 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.344245+0000", 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6462999999999998e-05 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.344606+0000", 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000361076 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.283 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345956+0000", 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001349442 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.349022+0000", 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030662419999999998 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.349092+0000", 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.9979000000000002e-05 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.284 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:886 3.ds0 3:bc68f0c8:::smithi10238019-15:head [append 0~0] snapc 1f=[1f,1e,1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.8447360350000004, 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089288609999999997, 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.285 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 886 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1699999999999999e-07 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340336+0000", 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3120000000000001e-06 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340342+0000", 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5590000000000002e-06 2024-09-17T17:30:48.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340347+0000", 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0719999999999999e-06 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340471+0000", 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012381499999999999 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340475+0000", 2024-09-17T17:30:48.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7270000000000001e-06 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345176+0000", 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047006569999999996 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345272+0000", 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6216000000000002e-05 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345702+0000", 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00043059000000000001 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.346307+0000", 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00060444100000000005 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.349198+0000", 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0028914650000000002 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.349262+0000", 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3590000000000006e-05 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 91/32 ECSubWrite(tid=166, reqid=osd.3.0:166, at_version=91'134, trim_to=0'0, roll_forward_to=85'132))", 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:42.349391+0000", 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 5.8356778829999998, 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015226195, 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349391+0000", 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349391+0000", 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.292 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349392+0000", 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349418+0000", 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6214e-05 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359149+0000", 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097303960000000005 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359157+0000", 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5350000000000004e-06 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359214+0000", 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6835e-05 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359223+0000", 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2019999999999993e-06 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.294 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.364582+0000", 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053591580000000002 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.364617+0000", 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5108999999999999e-05 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.295 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:48.297 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:48.297 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:48.294+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:48.346 INFO:tasks.rados.rados.0.smithi102.stdout:368: done (1 left) 2024-09-17T17:30:48.346 INFO:tasks.rados.rados.0.smithi102.stdout:371: done (0 left) 2024-09-17T17:30:48.346 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 30 current snap is 32 2024-09-17T17:30:48.346 INFO:tasks.rados.rados.0.smithi102.stdout:372: seq_num 158 ranges {2228224=262144} 2024-09-17T17:30:48.360 INFO:tasks.rados.rados.0.smithi102.stdout:372: writing smithi10238019-30 from 2228224 to 2490368 tid 1 2024-09-17T17:30:48.361 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 10 current snap is 32 2024-09-17T17:30:48.361 INFO:tasks.rados.rados.0.smithi102.stdout:373: seq_num 159 ranges {0=278528} 2024-09-17T17:30:48.363 INFO:tasks.rados.rados.0.smithi102.stdout:373: writing smithi10238019-10 from 0 to 278528 tid 1 2024-09-17T17:30:48.363 INFO:tasks.rados.rados.0.smithi102.stdout:374: read oid 46 snap -1 2024-09-17T17:30:48.363 INFO:tasks.rados.rados.0.smithi102.stdout:374: expect (ObjNum 113 snap 18 seq_num 113) 2024-09-17T17:30:48.363 INFO:tasks.rados.rados.0.smithi102.stdout:375: copy_from oid 43 from oid 17 current snap is 32 2024-09-17T17:30:48.363 INFO:tasks.rados.rados.0.smithi102.stdout:376: delete oid 36 current snap is 32 2024-09-17T17:30:48.371 INFO:tasks.rados.rados.0.smithi102.stdout:372: finishing write tid 1 to smithi10238019-30 2024-09-17T17:30:48.371 INFO:tasks.rados.rados.0.smithi102.stdout:372: finishing write tid 2 to smithi10238019-30 2024-09-17T17:30:48.372 INFO:tasks.rados.rados.0.smithi102.stdout:376: done (4 left) 2024-09-17T17:30:48.372 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 50 current snap is 32 2024-09-17T17:30:48.372 INFO:tasks.rados.rados.0.smithi102.stdout:377: seq_num 161 ranges {819200=311296} 2024-09-17T17:30:48.380 INFO:tasks.rados.rados.0.smithi102.stdout:377: writing smithi10238019-50 from 819200 to 1130496 tid 1 2024-09-17T17:30:48.380 INFO:tasks.rados.rados.0.smithi102.stdout:378: snap_remove snap 19 2024-09-17T17:30:48.398 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:48.396+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:48.418 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:48.419 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:48.419 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:48.419 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:48.428 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:48.499 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:48.497+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:48.531 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:48.531 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:48.531 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:48.531 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:48.531 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:48.540 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:48.600 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:48.599+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:48.644 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:48.644 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:48.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:48.644 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:48.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=24, last_complete=36'10, committed=1, applied=1))", 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 60.186693765000001, 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53983539199999997, 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:48.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454674+0000", 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0790000000000002e-06 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992637+0000", 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53796236600000003 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992644+0000", 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5820000000000004e-06 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994469+0000", 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018243350000000001 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994505+0000", 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6300999999999997e-05 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:174 3.cs0 3:30756a83:::smithi10238019-35:2 [read 501714~71726,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 59.651225672000002, 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011875784, 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.649 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 174 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9400000000000005e-07 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990142+0000", 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.557e-06 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990195+0000", 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3012e-05 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990206+0000", 2024-09-17T17:30:48.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0876000000000001e-05 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990783+0000", 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00057770200000000003 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990850+0000", 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6168999999999996e-05 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.002014+0000", 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011163874000000001 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:175 3.cs0 3:30756a83:::smithi10238019-35:2 [read 194830~378610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:48.654 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 59.651139114000003, 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.30022850299999998, 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.655 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 175 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990225+0000", 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8199999999999998e-07 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990230+0000", 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7809999999999996e-06 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990371+0000", 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014071299999999999 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990381+0000", 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0061000000000001e-05 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994702+0000", 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043214899999999999 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994769+0000", 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6600999999999995e-05 2024-09-17T17:30:48.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290453+0000", 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.29568397499999999 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.660 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=28, attrs_read=0))", 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 59.640166098000002, 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.289267987, 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001198+0000", 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6000000000000004e-07 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001279+0000", 2024-09-17T17:30:48.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1168000000000006e-05 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289843+0000", 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.28856357599999999 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289854+0000", 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1749999999999999e-05 2024-09-17T17:30:48.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289975+0000", 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012060100000000001 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290465+0000", 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049033200000000001 2024-09-17T17:30:48.664 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:316 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (56) in=7b] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 53.651905333000002, 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041035350999999998, 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.665 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 316 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.666 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989459+0000", 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2200000000000004e-07 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989462+0000", 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.287e-06 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989469+0000", 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7229999999999998e-06 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989472+0000", 2024-09-17T17:30:48.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.428e-06 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990430+0000", 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00095784099999999997 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990525+0000", 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5129000000000003e-05 2024-09-17T17:30:48.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990951+0000", 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042552399999999999 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991309+0000", 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000358632 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030441+0000", 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039131753999999998 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030493+0000", 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2510999999999999e-05 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:317 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 53.651871997000001, 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041694146000000001, 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.672 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 317 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:30:48.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989492+0000", 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3099999999999998e-07 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989494+0000", 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3329999999999999e-06 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989501+0000", 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.516e-06 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989505+0000", 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.912e-06 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991167+0000", 2024-09-17T17:30:48.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001662188 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991214+0000", 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6944000000000003e-05 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030733+0000", 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039519406999999999 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030782+0000", 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9338000000000001e-05 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.031186+0000", 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000403077 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.677 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:429 3.cs0 3:30d62bba:::smithi10238019-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.248460467000001, 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014137011, 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.678 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 429 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3299999999999998e-07 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392905+0000", 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6869999999999999e-06 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392912+0000", 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6250000000000001e-06 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392916+0000", 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7330000000000003e-06 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397110+0000", 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041939450000000001 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397209+0000", 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8961000000000003e-05 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402765+0000", 2024-09-17T17:30:48.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055556420000000004 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402840+0000", 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4943000000000004e-05 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.407040+0000", 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042001419999999996 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.683 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:436 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 12] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.248363568999999, 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012854290000000001, 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.684 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 436 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:48.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3700000000000001e-07 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393005+0000", 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4190000000000004e-06 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393008+0000", 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7009999999999998e-06 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393023+0000", 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4545999999999999e-05 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397903+0000", 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048805719999999997 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397964+0000", 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0520000000000003e-05 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.401036+0000", 2024-09-17T17:30:48.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030720470000000001 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402722+0000", 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001685513 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405761+0000", 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030398790000000001 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.689 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405854+0000", 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2756000000000004e-05 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:486 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 41.262357076000001, 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75510421100000003, 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.690 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 486 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379007+0000", 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379011+0000", 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0720000000000001e-06 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379019+0000", 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9580000000000007e-06 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379025+0000", 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6799999999999998e-06 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379992+0000", 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096721299999999997 2024-09-17T17:30:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380044+0000", 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2091e-05 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380358+0000", 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000313543 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380731+0000", 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00037292 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134034+0000", 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75330350700000004 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134111+0000", 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6497999999999998e-05 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.695 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:487 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 41.262327935000002, 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.756014517, 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.696 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 487 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.697 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379036+0000", 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5499999999999998e-07 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379038+0000", 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.698 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379045+0000", 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8770000000000004e-06 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379050+0000", 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2299999999999999e-06 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380522+0000", 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001471909 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.699 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380575+0000", 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2821999999999997e-05 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134417+0000", 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75384171300000002 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134482+0000", 2024-09-17T17:30:48.700 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5789000000000003e-05 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.135050+0000", 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00056751999999999996 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:626 3.cs0 3:30756a83:::smithi10238019-35:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:48.701 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.143066097999998, 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021079339999999998, 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 626 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.702 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498298+0000", 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.703 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498301+0000", 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7530000000000002e-06 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498307+0000", 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9869999999999996e-06 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498310+0000", 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1259999999999998e-06 2024-09-17T17:30:48.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499848+0000", 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001537912 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499903+0000", 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5763000000000001e-05 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500222+0000", 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00031825499999999998 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500848+0000", 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062612400000000004 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519301+0000", 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018453431999999999 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519377+0000", 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5405000000000001e-05 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:627 3.cs0 3:30756a83:::smithi10238019-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.143036938000002, 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021540693, 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 627 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498327+0000", 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.51e-07 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498329+0000", 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3989999999999998e-06 2024-09-17T17:30:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498337+0000", 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5379999999999997e-06 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498341+0000", 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4349999999999999e-06 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.710 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500569+0000", 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002227934 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500629+0000", 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9721e-05 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519409+0000", 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018780192000000001 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.711 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519465+0000", 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6062e-05 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519867+0000", 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000401961 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s6 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.140750633, 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26792300099999999, 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.713 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500614+0000", 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6700000000000002e-07 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.714 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500747+0000", 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013382499999999999 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767249+0000", 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26650138800000001 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767261+0000", 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1785e-05 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.715 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767372+0000", 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111743 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767387+0000", 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4548e-05 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768494+0000", 2024-09-17T17:30:48.716 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011074990000000001 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768536+0000", 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1346e-05 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:639 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:48.717 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.117568627000001, 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67002614400000005, 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 639 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.718 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.719 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0500000000000003e-07 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1729999999999999e-06 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523810+0000", 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1627000000000001e-05 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.720 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523815+0000", 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0420000000000002e-06 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525161+0000", 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0013460169999999999 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525253+0000", 2024-09-17T17:30:48.721 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1793999999999996e-05 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526004+0000", 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00075112400000000004 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527034+0000", 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001029713 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.722 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193749+0000", 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.66671533999999999 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193821+0000", 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1709000000000001e-05 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:640 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:30:48.723 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.117537011, 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67078859899999999, 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 640 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.724 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523827+0000", 2024-09-17T17:30:48.725 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.03e-07 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523829+0000", 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1739999999999999e-06 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523836+0000", 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3869999999999997e-06 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.726 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523841+0000", 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4089999999999996e-06 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527242+0000", 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003401383 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527298+0000", 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5705000000000003e-05 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.727 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194159+0000", 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.666861484 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194218+0000", 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9039000000000002e-05 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194615+0000", 2024-09-17T17:30:48.728 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039661500000000001 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 68/32 ECSubWrite(tid=118, reqid=client.4566.0:641, at_version=68'91, trim_to=0'0, roll_forward_to=66'89))", 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.116695894999999, 2024-09-17T17:30:48.729 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.164368816, 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:48.730 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524694+0000", 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6256000000000001e-05 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.731 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687716+0000", 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.163021575 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687725+0000", 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9390000000000003e-06 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687798+0000", 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900999999999999e-05 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.732 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687809+0000", 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.153e-05 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689007+0000", 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001197844 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.733 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689036+0000", 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9342000000000001e-05 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 78/32 ECSubWrite(tid=154, reqid=osd.3.0:154, at_version=78'120, trim_to=0'0, roll_forward_to=78'118))", 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.384978544999999, 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.088487576999999998, 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.734 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.735 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256386+0000", 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3900000000000004e-07 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256409+0000", 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3475000000000001e-05 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342362+0000", 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.085952671999999994 2024-09-17T17:30:48.736 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342372+0000", 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0285000000000001e-05 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343593+0000", 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001220858 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.737 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343604+0000", 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1078e-05 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344835+0000", 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0012308309999999999 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344872+0000", 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7539000000000001e-05 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.738 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 91/87 ECSubWrite(tid=84, reqid=osd.14.0:84, at_version=91'76, trim_to=0'0, roll_forward_to=89'72))", 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 6.2873367289999997, 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.14743911800000001, 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.739 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.740 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354048+0000", 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0214999999999999e-05 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500785+0000", 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.146736905 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500799+0000", 2024-09-17T17:30:48.741 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4887e-05 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500878+0000", 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8880000000000004e-05 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500886+0000", 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5270000000000001e-06 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.742 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.501437+0000", 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000551134 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.501466+0000", 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8836e-05 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:941 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 75] snapc 21=[21,20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.279288547, 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.020459694, 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 941 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.744 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.745 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362077+0000", 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.984e-06 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362089+0000", 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2456000000000001e-05 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362107+0000", 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8053000000000001e-05 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.746 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362119+0000", 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2055999999999999e-05 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362183+0000", 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3246999999999999e-05 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2470000000000002e-06 2024-09-17T17:30:48.747 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364142+0000", 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0019523780000000001 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364249+0000", 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010689899999999999 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.377910+0000", 2024-09-17T17:30:48.748 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013660960999999999 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.380133+0000", 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0022232409999999999 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.382430+0000", 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00229707 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.749 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.382535+0000", 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000104102 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:942 3.cs0 3:349443fd:::smithi10238019-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 0.27921991400000001, 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.017198940999999999, 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:48.750 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 942 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.751 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.96e-07 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362147+0000", 2024-09-17T17:30:48.752 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5219999999999999e-06 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362153+0000", 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8159999999999999e-06 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362157+0000", 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.245e-06 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.753 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362251+0000", 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3163000000000002e-05 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362255+0000", 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1849999999999997e-06 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364403+0000", 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002147726 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.754 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364448+0000", 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5776999999999999e-05 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379154+0000", 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014705371 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379256+0000", 2024-09-17T17:30:48.755 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000102272 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379342+0000", 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6468e-05 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:48.756 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:48.757 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 2] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:30:48.757 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:48.757 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T17:30:48.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T17:30:48.759 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:48.759 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:48.756+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:48.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:48.859 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:48.856+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:48.902 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:48.903 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:48.903 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:30:48.903 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:48.911 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:48.946 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:48.960 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:48.958+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:48.969 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:48.969 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:48.990 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:48.991 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:49.011 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:49.013 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:49.014 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:30:49.014 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:30:49.014 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:30:49.014 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:49.022 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:49.032 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:49.032 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:49.053 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:49.061 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:49.059+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:49.075 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:49.085 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 97, 2024-09-17T17:30:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "2593b77f-a53d-4649-b5a5-0122bd31e2b3", 2024-09-17T17:30:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T17:29:15.547925+0000", 2024-09-17T17:30:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T17:30:48.340391+0000", 2024-09-17T17:30:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T17:29:28.558396+0000", 2024-09-17T17:30:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T17:30:22.332815+0000", 2024-09-17T17:30:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T17:30:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5804032, 2024-09-17T17:30:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T17:30:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-17T17:30:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T17:30:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T17:30:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T17:30:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T17:30:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 7, 2024-09-17T17:30:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T17:30:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T17:30:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T17:30:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T17:30:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T17:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 16, 2024-09-17T17:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T17:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-09-17T17:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T17:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T17:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T17:30:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T17:30:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T17:30:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T17:29:27.893486+0000", 2024-09-17T17:30:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T17:30:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T17:30:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T17:30:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T17:30:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T17:30:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T17:30:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T17:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T17:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T17:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T17:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T17:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T17:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T17:30:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T17:30:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T17:30:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T17:30:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T17:30:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T17:30:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T17:30:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T17:30:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T17:30:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T17:30:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T17:30:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T17:30:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T17:30:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "25", 2024-09-17T17:30:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T17:30:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T17:30:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T17:30:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T17:30:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T17:30:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T17:30:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T17:30:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T17:30:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T17:30:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T17:30:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T17:30:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T17:30:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T17:30:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T17:30:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T17:30:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T17:30:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T17:30:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T17:30:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T17:30:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T17:30:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T17:30:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T17:30:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T17:30:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T17:30:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T17:30:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T17:30:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 15.380000114440918, 2024-09-17T17:30:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 15.380000114440918, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T17:30:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T17:30:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T17:29:30.150043+0000", 2024-09-17T17:30:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T17:30:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T17:30:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T17:30:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T17:30:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "28", 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 28, 2024-09-17T17:30:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T17:30:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T17:30:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T17:30:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 4.0199999809265137, 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 4.0199999809265137, 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.56000000238418579, 2024-09-17T17:30:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "unique_pool_0", 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T17:29:42.633017+0000", 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T17:30:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 3, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 7, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 5, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 1, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 26, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 20, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 26, 2024-09-17T17:30:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 26, 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 26, 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "97", 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T17:30:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "67", 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 33, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 97, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T17:30:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "shecprofile", 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T17:30:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 16384, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T17:30:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "rados": {} 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "72ee94e9-57bf-46b3-9fb9-0d05515fe1e1", 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 0, 2024-09-17T17:30:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T17:30:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/50344815", 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/50344815", 2024-09-17T17:30:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/50344815", 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/50344815", 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e9764092-a458-4b6d-8bee-658f54f000cc", 2024-09-17T17:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.68646240234375, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 21, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 94, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6816", 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6817", 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6818", 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6819", 2024-09-17T17:30:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6822", 2024-09-17T17:30:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6823", 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6820", 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6821", 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6817/2426784045", 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6819/2426784045", 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6823/2426784045", 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6821/2426784045", 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T17:30:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f1142f7d-9945-4daa-8f6a-6fd2d3090b5f", 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 0, 2024-09-17T17:30:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6800", 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6801", 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6802", 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6803", 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6806", 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6807", 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6804", 2024-09-17T17:30:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6805", 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6801/1502510538", 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6803/1502510538", 2024-09-17T17:30:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6807/1502510538", 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6805/1502510538", 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "daa4b149-c71a-4cba-a710-89aa101d7d96", 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6810", 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6811", 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6812", 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6813", 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6816", 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6817", 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6814", 2024-09-17T17:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6815", 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6811/2453627254", 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6813/2453627254", 2024-09-17T17:30:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6817/2453627254", 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6815/2453627254", 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "55e8e034-493b-4066-a42b-f4d82294f3f8", 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 84, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/3507123258", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/3507123258", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/3507123258", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/3507123258", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "38a09763-d370-447a-b939-bad8a11c5e01", 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 0, 2024-09-17T17:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6824", 2024-09-17T17:30:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6825", 2024-09-17T17:30:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6826", 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6827", 2024-09-17T17:30:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6830", 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6831", 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6828", 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6829", 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6825/229014119", 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6827/229014119", 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6831/229014119", 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6829/229014119", 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "0ae069f9-8831-4bfc-9928-2100b73a839d", 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6816", 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6817", 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6818", 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6819", 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6822", 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6823", 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6820", 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6821", 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6817/3536708597", 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6819/3536708597", 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6823/3536708597", 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6821/3536708597", 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b", 2024-09-17T17:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 21, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 88, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6802", 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6803", 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6804", 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6805", 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6808", 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6809", 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6806", 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6807", 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6803/2367087286", 2024-09-17T17:30:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6805/2367087286", 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6809/2367087286", 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6807/2367087286", 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 8, 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "3c4daae0-6d93-450f-8600-b29362bd9f77", 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 32, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T17:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/1273991967", 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/1273991967", 2024-09-17T17:30:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1273991967", 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1273991967", 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 9, 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2846b3c6-06d6-414f-b4c9-b16d94007737", 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 25, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6808", 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6809", 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6810", 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6811", 2024-09-17T17:30:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6814", 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6815", 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6812", 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6813", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6809/2309011334", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6811/2309011334", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6815/2309011334", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6813/2309011334", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 10, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "495d7d7c-c222-4a46-9685-2a2a7cc6fc51", 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 74, 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6808", 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6809", 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6810", 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6811", 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6814", 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6815", 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6812", 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6813", 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6809/2168900921", 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6811/2168900921", 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6815/2168900921", 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6813/2168900921", 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 11, 2024-09-17T17:30:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf76d383-65cd-48c3-a54c-cd83ffbc9898", 2024-09-17T17:30:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6826", 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6827", 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6828", 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6829", 2024-09-17T17:30:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6832", 2024-09-17T17:30:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6833", 2024-09-17T17:30:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6830", 2024-09-17T17:30:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6831", 2024-09-17T17:30:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6827/3109125505", 2024-09-17T17:30:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6829/3109125505", 2024-09-17T17:30:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6833/3109125505", 2024-09-17T17:30:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6831/3109125505", 2024-09-17T17:30:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 12, 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "54294878-ea1e-4845-a0e6-9be026f521fa", 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.74237060546875, 2024-09-17T17:30:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 76, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-17T17:30:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-17T17:30:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/3659919075", 2024-09-17T17:30:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/3659919075", 2024-09-17T17:30:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/3659919075", 2024-09-17T17:30:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/3659919075", 2024-09-17T17:30:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 13, 2024-09-17T17:30:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "9acf83a5-75db-4874-8f3f-7421cc55e2fd", 2024-09-17T17:30:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.8008880615234375, 2024-09-17T17:30:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6800", 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6801", 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6802", 2024-09-17T17:30:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6803", 2024-09-17T17:30:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6806", 2024-09-17T17:30:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6807", 2024-09-17T17:30:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6804", 2024-09-17T17:30:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6805", 2024-09-17T17:30:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6801/1983765258", 2024-09-17T17:30:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6803/1983765258", 2024-09-17T17:30:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6807/1983765258", 2024-09-17T17:30:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6805/1983765258", 2024-09-17T17:30:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 14, 2024-09-17T17:30:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "16c02ebe-a418-4f87-902d-b5dac864f139", 2024-09-17T17:30:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 94, 2024-09-17T17:30:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6824", 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6825", 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6826", 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6827", 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6830", 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6831", 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6828", 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6829", 2024-09-17T17:30:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6825/1199123829", 2024-09-17T17:30:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6827/1199123829", 2024-09-17T17:30:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6831/1199123829", 2024-09-17T17:30:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6829/1199123829", 2024-09-17T17:30:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 15, 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa", 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 94, 2024-09-17T17:30:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6818", 2024-09-17T17:30:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6819", 2024-09-17T17:30:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6820", 2024-09-17T17:30:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6821", 2024-09-17T17:30:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6824", 2024-09-17T17:30:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6825", 2024-09-17T17:30:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6822", 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6823", 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6819/259571866", 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6821/259571866", 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6825/259571866", 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6823/259571866", 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:29:25.950873+0000", 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:27.557824+0000", 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:29:25.560618+0000", 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:44.107535+0000", 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:43.930932+0000", 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:46.730260+0000", 2024-09-17T17:30:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-17T17:30:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:43.862104+0000", 2024-09-17T17:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-17T17:30:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:42.595312+0000", 2024-09-17T17:30:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 8, 2024-09-17T17:30:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:40.607805+0000", 2024-09-17T17:30:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 9, 2024-09-17T17:30:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:40.265380+0000", 2024-09-17T17:30:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 10, 2024-09-17T17:30:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:33.448841+0000", 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 11, 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:29.957316+0000", 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 12, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:29.888647+0000", 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 13, 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:29:25.757133+0000", 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 14, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:29:26.479198+0000", 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 15, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:32.994404+0000", 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T17:30:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "shecprofile": { 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "c": "2", 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-device-class": "", 2024-09-17T17:30:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-num-failure-domains": "0", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-osds-per-failure-domain": "0", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-root": "default", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "4", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "3", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "shecprofile", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "shec", 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "multiple" 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 19, 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 25, 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 7 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-17T17:30:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:49.284 INFO:tasks.thrashosds.thrasher:j is {'epoch': 97, 'fsid': '2593b77f-a53d-4649-b5a5-0122bd31e2b3', 'created': '2024-09-17T17:29:15.547925+0000', 'modified': '2024-09-17T17:30:48.340391+0000', 'last_up_change': '2024-09-17T17:29:28.558396+0000', 'last_in_change': '2024-09-17T17:30:22.332815+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T17:29:27.893486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T17:29:30.150043+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-17T17:29:42.633017+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 20, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '67', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '72ee94e9-57bf-46b3-9fb9-0d05515fe1e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 50344815}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 50344815}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 50344815}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 50344815}]}, 'public_addr': '172.21.15.7:6817/50344815', 'cluster_addr': '172.21.15.7:6819/50344815', 'heartbeat_back_addr': '172.21.15.7:6823/50344815', 'heartbeat_front_addr': '172.21.15.7:6821/50344815', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e9764092-a458-4b6d-8bee-658f54f000cc', 'up': 1, 'in': 1, 'weight': 0.68646240234375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6816', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6817', 'nonce': 2426784045}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6818', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6819', 'nonce': 2426784045}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6822', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6823', 'nonce': 2426784045}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6820', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6821', 'nonce': 2426784045}]}, 'public_addr': '172.21.15.26:6817/2426784045', 'cluster_addr': '172.21.15.26:6819/2426784045', 'heartbeat_back_addr': '172.21.15.26:6823/2426784045', 'heartbeat_front_addr': '172.21.15.26:6821/2426784045', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f1142f7d-9945-4daa-8f6a-6fd2d3090b5f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 1502510538}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 1502510538}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 1502510538}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 1502510538}]}, 'public_addr': '172.21.15.27:6801/1502510538', 'cluster_addr': '172.21.15.27:6803/1502510538', 'heartbeat_back_addr': '172.21.15.27:6807/1502510538', 'heartbeat_front_addr': '172.21.15.27:6805/1502510538', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'daa4b149-c71a-4cba-a710-89aa101d7d96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6810', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6811', 'nonce': 2453627254}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6812', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6813', 'nonce': 2453627254}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6816', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 2453627254}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6814', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6815', 'nonce': 2453627254}]}, 'public_addr': '172.21.15.102:6811/2453627254', 'cluster_addr': '172.21.15.102:6813/2453627254', 'heartbeat_back_addr': '172.21.15.102:6817/2453627254', 'heartbeat_front_addr': '172.21.15.102:6815/2453627254', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '55e8e034-493b-4066-a42b-f4d82294f3f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 3507123258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 3507123258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 3507123258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 3507123258}]}, 'public_addr': '172.21.15.7:6801/3507123258', 'cluster_addr': '172.21.15.7:6803/3507123258', 'heartbeat_back_addr': '172.21.15.7:6807/3507123258', 'heartbeat_front_addr': '172.21.15.7:6805/3507123258', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '38a09763-d370-447a-b939-bad8a11c5e01', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6824', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6825', 'nonce': 229014119}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6826', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6827', 'nonce': 229014119}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6830', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6831', 'nonce': 229014119}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6828', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6829', 'nonce': 229014119}]}, 'public_addr': '172.21.15.26:6825/229014119', 'cluster_addr': '172.21.15.26:6827/229014119', 'heartbeat_back_addr': '172.21.15.26:6831/229014119', 'heartbeat_front_addr': '172.21.15.26:6829/229014119', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '0ae069f9-8831-4bfc-9928-2100b73a839d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 3536708597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 3536708597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 3536708597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 3536708597}]}, 'public_addr': '172.21.15.27:6817/3536708597', 'cluster_addr': '172.21.15.27:6819/3536708597', 'heartbeat_back_addr': '172.21.15.27:6823/3536708597', 'heartbeat_front_addr': '172.21.15.27:6821/3536708597', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6802', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6803', 'nonce': 2367087286}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6804', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6805', 'nonce': 2367087286}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6808', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6809', 'nonce': 2367087286}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6806', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6807', 'nonce': 2367087286}]}, 'public_addr': '172.21.15.102:6803/2367087286', 'cluster_addr': '172.21.15.102:6805/2367087286', 'heartbeat_back_addr': '172.21.15.102:6809/2367087286', 'heartbeat_front_addr': '172.21.15.102:6807/2367087286', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '3c4daae0-6d93-450f-8600-b29362bd9f77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 1273991967}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1273991967}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1273991967}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1273991967}]}, 'public_addr': '172.21.15.7:6809/1273991967', 'cluster_addr': '172.21.15.7:6811/1273991967', 'heartbeat_back_addr': '172.21.15.7:6815/1273991967', 'heartbeat_front_addr': '172.21.15.7:6813/1273991967', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '2846b3c6-06d6-414f-b4c9-b16d94007737', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6808', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6809', 'nonce': 2309011334}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6810', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6811', 'nonce': 2309011334}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6814', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6815', 'nonce': 2309011334}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6812', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6813', 'nonce': 2309011334}]}, 'public_addr': '172.21.15.26:6809/2309011334', 'cluster_addr': '172.21.15.26:6811/2309011334', 'heartbeat_back_addr': '172.21.15.26:6815/2309011334', 'heartbeat_front_addr': '172.21.15.26:6813/2309011334', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '495d7d7c-c222-4a46-9685-2a2a7cc6fc51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 2168900921}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 2168900921}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 2168900921}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 2168900921}]}, 'public_addr': '172.21.15.27:6809/2168900921', 'cluster_addr': '172.21.15.27:6811/2168900921', 'heartbeat_back_addr': '172.21.15.27:6815/2168900921', 'heartbeat_front_addr': '172.21.15.27:6813/2168900921', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'cf76d383-65cd-48c3-a54c-cd83ffbc9898', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6826', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6827', 'nonce': 3109125505}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6828', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6829', 'nonce': 3109125505}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 3109125505}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6830', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6831', 'nonce': 3109125505}]}, 'public_addr': '172.21.15.102:6827/3109125505', 'cluster_addr': '172.21.15.102:6829/3109125505', 'heartbeat_back_addr': '172.21.15.102:6833/3109125505', 'heartbeat_front_addr': '172.21.15.102:6831/3109125505', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '54294878-ea1e-4845-a0e6-9be026f521fa', 'up': 1, 'in': 1, 'weight': 0.74237060546875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 3659919075}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 3659919075}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 3659919075}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 3659919075}]}, 'public_addr': '172.21.15.7:6825/3659919075', 'cluster_addr': '172.21.15.7:6827/3659919075', 'heartbeat_back_addr': '172.21.15.7:6831/3659919075', 'heartbeat_front_addr': '172.21.15.7:6829/3659919075', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '9acf83a5-75db-4874-8f3f-7421cc55e2fd', 'up': 1, 'in': 1, 'weight': 0.8008880615234375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6800', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6801', 'nonce': 1983765258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6802', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6803', 'nonce': 1983765258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6806', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6807', 'nonce': 1983765258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6804', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6805', 'nonce': 1983765258}]}, 'public_addr': '172.21.15.26:6801/1983765258', 'cluster_addr': '172.21.15.26:6803/1983765258', 'heartbeat_back_addr': '172.21.15.26:6807/1983765258', 'heartbeat_front_addr': '172.21.15.26:6805/1983765258', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '16c02ebe-a418-4f87-902d-b5dac864f139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6824', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6825', 'nonce': 1199123829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6826', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6827', 'nonce': 1199123829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6830', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6831', 'nonce': 1199123829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6828', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6829', 'nonce': 1199123829}]}, 'public_addr': '172.21.15.27:6825/1199123829', 'cluster_addr': '172.21.15.27:6827/1199123829', 'heartbeat_back_addr': '172.21.15.27:6831/1199123829', 'heartbeat_front_addr': '172.21.15.27:6829/1199123829', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': 'a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6818', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6819', 'nonce': 259571866}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6820', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6821', 'nonce': 259571866}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6824', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6825', 'nonce': 259571866}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6822', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6823', 'nonce': 259571866}]}, 'public_addr': '172.21.15.102:6819/259571866', 'cluster_addr': '172.21.15.102:6821/259571866', 'heartbeat_back_addr': '172.21.15.102:6825/259571866', 'heartbeat_front_addr': '172.21.15.102:6823/259571866', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:29:25.950873+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:27.557824+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:29:25.560618+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:44.107535+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:43.930932+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:46.730260+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:43.862104+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:42.595312+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:40.607805+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:40.265380+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:33.448841+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:29.957316+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:29.888647+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:29:25.757133+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:29:26.479198+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:32.994404+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 19, 'length': 1}, {'begin': 25, 'length': 7}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T17:30:49.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:30:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:49.285 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:30:49.285 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:30:49.285 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:30:49.285 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:30:49.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 37/35 ECSubWriteReply(tid=17, last_complete=37'13, committed=1, applied=1))", 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 60.163235325999999, 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.068701408000000005, 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.286 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000937+0000", 2024-09-17T17:30:49.287 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6449999999999998e-06 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068896+0000", 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.067959031000000003 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068909+0000", 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3416e-05 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.288 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069008+0000", 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8690999999999994e-05 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069634+0000", 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062589099999999995 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.289 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 58.480304068000002, 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079875469999999994, 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.290 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2e-07 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684336+0000", 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047134800000000001 2024-09-17T17:30:49.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684439+0000", 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010291800000000001 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684449+0000", 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.027e-05 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.292 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684596+0000", 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000146749 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684607+0000", 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1421e-05 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691787+0000", 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071804069999999998 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.293 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691851+0000", 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3913999999999998e-05 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 58.478813662, 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089994210000000005, 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.294 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.295 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4100000000000001e-07 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685373+0000", 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9063999999999999e-05 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687383+0000", 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002009355 2024-09-17T17:30:49.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687391+0000", 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9699999999999999e-06 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687457+0000", 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5834999999999996e-05 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.297 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9990000000000003e-06 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694321+0000", 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068546010000000001 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694354+0000", 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2456000000000002e-05 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.298 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:191 3.5s0 3:a4d61510:::smithi10238019-32:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 58.476730799999999, 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.008961732, 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.299 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 191 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:30:49.300 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687439+0000", 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1799999999999999e-06 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687504+0000", 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4770000000000005e-05 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.301 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687516+0000", 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2007e-05 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687531+0000", 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4803e-05 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687595+0000", 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3697999999999999e-05 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.302 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687756+0000", 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000161846 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.690070+0000", 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0023137639999999998 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695782+0000", 2024-09-17T17:30:49.303 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057116959999999996 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695908+0000", 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000126339 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.696399+0000", 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049062900000000002 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:407 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.826090651000001, 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011701008000000001, 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 407 2024-09-17T17:30:49.305 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.306 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338080+0000", 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2819999999999999e-06 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6289999999999999e-06 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338108+0000", 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9514e-05 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.307 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338117+0000", 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.132e-06 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1446999999999995e-05 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338196+0000", 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6350000000000006e-06 2024-09-17T17:30:49.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341655+0000", 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034593860000000001 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342199+0000", 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054402400000000005 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.309 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349778+0000", 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075789589999999997 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:408 3.5s0 3:a96b502c:::smithi10238019-42:head [read 53220~651292,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.825928724000001, 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012064669, 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.310 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 408 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.311 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338240+0000", 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338257+0000", 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6643e-05 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.312 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338270+0000", 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3081000000000001e-05 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338277+0000", 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3780000000000003e-06 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338331+0000", 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4286999999999997e-05 2024-09-17T17:30:49.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338338+0000", 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3609999999999997e-06 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342681+0000", 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043432749999999997 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.314 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342775+0000", 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4392e-05 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.350304+0000", 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007528435 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:409 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:49.315 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.825815968999997, 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012665852, 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 409 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.316 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338353+0000", 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.09e-07 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.317 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338358+0000", 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7859999999999996e-06 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338366+0000", 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6049999999999998e-06 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338372+0000", 2024-09-17T17:30:49.318 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1739999999999997e-06 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338439+0000", 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7039999999999995e-05 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338446+0000", 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1160000000000001e-06 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.319 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343114+0000", 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0046672179999999999 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343207+0000", 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3170000000000004e-05 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351018+0000", 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078109340000000003 2024-09-17T17:30:49.320 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 46.811661589000003, 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.988933536, 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.321 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.322 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352507+0000", 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1500000000000003e-07 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352532+0000", 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4485999999999999e-05 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340513+0000", 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.98798163100000003 2024-09-17T17:30:49.323 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340522+0000", 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9379999999999998e-06 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340625+0000", 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010273 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340633+0000", 2024-09-17T17:30:49.324 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341398+0000", 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00076522600000000004 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341440+0000", 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1548999999999998e-05 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.325 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/35 ECSubWrite(tid=50, reqid=client.4566.0:441, at_version=0'0, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 45.300604069000002, 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.121580543, 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:49.326 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863565+0000", 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6729999999999999e-06 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.327 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863588+0000", 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2756e-05 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984520+0000", 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.120931489 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984547+0000", 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6877999999999999e-05 2024-09-17T17:30:49.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984631+0000", 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4246000000000006e-05 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984638+0000", 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3509999999999996e-06 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.329 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985117+0000", 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047851700000000003 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985144+0000", 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7633000000000001e-05 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:49.330 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 29.663513536, 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.090576745, 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.331 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500656+0000", 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0430000000000002e-06 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500772+0000", 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000115338 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.332 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590079+0000", 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089307485000000006 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590103+0000", 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4131e-05 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590159+0000", 2024-09-17T17:30:49.333 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5417000000000002e-05 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590177+0000", 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7819e-05 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591201+0000", 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023939 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.334 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591231+0000", 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0573e-05 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 75/74 ECSubWrite(tid=94, reqid=osd.10.0:94, at_version=75'58, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.812045909999998, 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086672569999999994, 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.335 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.336 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352123+0000", 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352143+0000", 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0440000000000001e-05 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352170+0000", 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6503e-05 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.337 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352177+0000", 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6429999999999995e-06 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352236+0000", 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8711000000000001e-05 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352245+0000", 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:30:49.338 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360755+0000", 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085095469999999993 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360789+0000", 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4678999999999998e-05 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.339 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 78/74 ECSubWrite(tid=102, reqid=client.4566.0:771, at_version=78'64, trim_to=0'0, roll_forward_to=78'60))", 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 19.782314569, 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96796263999999999, 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.340 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381854+0000", 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4e-07 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.341 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381870+0000", 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6076e-05 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348683+0000", 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96681319499999996 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348717+0000", 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.342 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348821+0000", 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010401800000000001 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348833+0000", 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2172e-05 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349780+0000", 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00094660899999999995 2024-09-17T17:30:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349816+0000", 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6025999999999997e-05 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 79/74 ECSubWrite(tid=108, reqid=client.4566.0:788, at_version=79'60, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 18.798608517000002, 2024-09-17T17:30:49.344 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010065843999999999, 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:30:49.345 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365560+0000", 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5199999999999995e-07 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365578+0000", 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8145999999999999e-05 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.346 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374667+0000", 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.009088245 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374697+0000", 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0402000000000001e-05 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374763+0000", 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5912000000000002e-05 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.347 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374776+0000", 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2497000000000001e-05 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375591+0000", 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00081505399999999995 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375625+0000", 2024-09-17T17:30:49.348 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4635999999999998e-05 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 80/74 ECSubWrite(tid=112, reqid=osd.10.0:111, at_version=0'0, trim_to=0'0, roll_forward_to=80'62))", 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.811784685999999, 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076437279999999998, 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.349 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352384+0000", 2024-09-17T17:30:49.350 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0249999999999999e-06 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352400+0000", 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5628999999999999e-05 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352437+0000", 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6813000000000001e-05 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.351 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352448+0000", 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0882e-05 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352672+0000", 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00022395999999999999 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352683+0000", 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1656e-05 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.352 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.359996+0000", 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073125120000000002 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.360027+0000", 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1251000000000003e-05 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.353 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.13.22:9 3.5s0 3:a4d61510:::smithi10238019-32:head [assert-version v27,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.822077276, 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015638470000000002, 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.354 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342092+0000", 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7e-06 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.355 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342098+0000", 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8690000000000001e-06 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342114+0000", 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5954999999999999e-05 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342122+0000", 2024-09-17T17:30:49.356 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3919999999999997e-06 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342179+0000", 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7768e-05 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342187+0000", 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3819999999999998e-06 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.357 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348515+0000", 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063278780000000003 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348672+0000", 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015732800000000001 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.357729+0000", 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090571980000000007 2024-09-17T17:30:49.358 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 85/76 ECSubWrite(tid=194, reqid=osd.12.0:193, at_version=0'0, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.81186484, 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0091352759999999995, 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.359 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352304+0000", 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4e-07 2024-09-17T17:30:49.360 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352316+0000", 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1864e-05 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352360+0000", 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4478999999999999e-05 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.361 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352370+0000", 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5829999999999996e-06 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352437+0000", 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7521000000000006e-05 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352447+0000", 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2180000000000005e-06 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.362 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361372+0000", 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089248839999999992 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361438+0000", 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6786999999999996e-05 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.363 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 85/74 ECSubWrite(tid=125, reqid=osd.10.0:125, at_version=85'76, trim_to=0'0, roll_forward_to=83'74))", 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 12.811230505999999, 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085212919999999998, 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.364 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352938+0000", 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.04e-07 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352958+0000", 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0486e-05 2024-09-17T17:30:49.365 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353044+0000", 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5852000000000004e-05 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353050+0000", 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2280000000000003e-06 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353110+0000", 2024-09-17T17:30:49.366 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9478999999999998e-05 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353121+0000", 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1426e-05 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361426+0000", 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0083047659999999999 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.367 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361459+0000", 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2851000000000001e-05 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:922 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.8186547630000001, 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010998030000000001, 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.368 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 922 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.369 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345515+0000", 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4219999999999999e-06 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345520+0000", 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2909999999999998e-06 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.370 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345543+0000", 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3323e-05 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345551+0000", 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3549999999999999e-06 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345653+0000", 2024-09-17T17:30:49.371 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010261500000000001 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345661+0000", 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1310000000000006e-06 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.352634+0000", 2024-09-17T17:30:49.372 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0069729299999999996 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.352757+0000", 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000122714 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.356511+0000", 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0037542489999999999 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.373 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:923 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.8186003880000001, 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011747277, 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.374 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 923 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.375 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345568+0000", 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6300000000000005e-07 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345574+0000", 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5219999999999998e-06 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345580+0000", 2024-09-17T17:30:49.376 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1120000000000001e-06 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345585+0000", 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3229999999999997e-06 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345814+0000", 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000228989 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.377 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345820+0000", 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2419999999999997e-06 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.353324+0000", 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075038520000000001 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.353431+0000", 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107129 2024-09-17T17:30:49.378 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.357315+0000", 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003883545 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:924 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 1.818439795, 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012050959, 2024-09-17T17:30:49.379 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 924 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.380 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345729+0000", 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.015e-06 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345734+0000", 2024-09-17T17:30:49.381 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2909999999999998e-06 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345745+0000", 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0134e-05 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345760+0000", 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5775999999999999e-05 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.382 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345844+0000", 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3635999999999997e-05 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345849+0000", 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2100000000000001e-06 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.354091+0000", 2024-09-17T17:30:49.383 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0082413780000000006 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.354183+0000", 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2315e-05 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.357779+0000", 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003596204 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.384 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:30:49.385 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:30:49.385 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:30:49.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:49.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:49.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:49.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:49.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:30:49.387 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:49.388 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:49.280+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:49.388 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:378: done (5 left) 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:379: delete oid 48 current snap is 32 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:372: finishing write tid 3 to smithi10238019-30 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:372: oid 30 updating version 0 to 154 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:372: oid 30 updating version 154 to 155 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 30 v 155 (ObjNum 158 snap 32 seq_num 158) dirty exists 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:372: left oid 30 (ObjNum 158 snap 32 seq_num 158) 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:373: finishing write tid 1 to smithi10238019-10 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:373: finishing write tid 2 to smithi10238019-10 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:373: finishing write tid 3 to smithi10238019-10 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:373: oid 10 updating version 0 to 139 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:373: oid 10 version 139 is already newer than 138 2024-09-17T17:30:49.389 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 10 v 139 (ObjNum 159 snap 32 seq_num 159) dirty exists 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:373: left oid 10 (ObjNum 159 snap 32 seq_num 159) 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:375: finishing copy_from racing read to smithi10238019-43 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:375: finishing copy_from to smithi10238019-43 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 43 v 75 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:377: finishing write tid 1 to smithi10238019-50 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:377: finishing write tid 2 to smithi10238019-50 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:377: finishing write tid 3 to smithi10238019-50 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:377: oid 50 updating version 0 to 167 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:377: oid 50 version 167 is already newer than 166 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 50 v 167 (ObjNum 161 snap 32 seq_num 161) dirty exists 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:377: left oid 50 (ObjNum 161 snap 32 seq_num 161) 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:374: expect (ObjNum 113 snap 18 seq_num 113) 2024-09-17T17:30:49.390 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 2143054 2024-09-17T17:30:49.391 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:49.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:49.391 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:49.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:49.392 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:49.246+0000 7faa8597d640 0 -- 172.21.15.7:0/1065639708 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7faa80221680 msgr2=0x7faa80223a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:49.393 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:49.393 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:49.393 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:49.385+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:49.394 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:49.414 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:49.489 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:49.488+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:49.498 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1077811 2024-09-17T17:30:49.546 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:49.546 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:49.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:49.564 INFO:tasks.rados.rados.0.smithi102.stdout:372: done (5 left) 2024-09-17T17:30:49.564 INFO:tasks.rados.rados.0.smithi102.stdout:373: done (4 left) 2024-09-17T17:30:49.564 INFO:tasks.rados.rados.0.smithi102.stdout:374: done (3 left) 2024-09-17T17:30:49.564 INFO:tasks.rados.rados.0.smithi102.stdout:375: done (2 left) 2024-09-17T17:30:49.564 INFO:tasks.rados.rados.0.smithi102.stdout:377: done (1 left) 2024-09-17T17:30:49.565 INFO:tasks.rados.rados.0.smithi102.stdout:379: done (0 left) 2024-09-17T17:30:49.565 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 42 current snap is 32 2024-09-17T17:30:49.565 INFO:tasks.rados.rados.0.smithi102.stdout:380: seq_num 163 ranges {0=344064} 2024-09-17T17:30:49.566 INFO:tasks.rados.rados.0.smithi102.stdout:380: writing smithi10238019-42 from 0 to 344064 tid 1 2024-09-17T17:30:49.566 INFO:tasks.rados.rados.0.smithi102.stdout:381: snap_create 2024-09-17T17:30:49.569 INFO:tasks.rados.rados.0.smithi102.stdout:380: finishing write tid 1 to smithi10238019-42 2024-09-17T17:30:49.570 INFO:tasks.rados.rados.0.smithi102.stdout:380: finishing write tid 2 to smithi10238019-42 2024-09-17T17:30:49.572 INFO:tasks.rados.rados.0.smithi102.stdout:380: finishing write tid 3 to smithi10238019-42 2024-09-17T17:30:49.572 INFO:tasks.rados.rados.0.smithi102.stdout:380: oid 42 updating version 0 to 36 2024-09-17T17:30:49.572 INFO:tasks.rados.rados.0.smithi102.stdout:380: oid 42 version 36 is already newer than 35 2024-09-17T17:30:49.572 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 42 v 36 (ObjNum 163 snap 32 seq_num 163) dirty exists 2024-09-17T17:30:49.572 INFO:tasks.rados.rados.0.smithi102.stdout:380: left oid 42 (ObjNum 163 snap 32 seq_num 163) 2024-09-17T17:30:49.590 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:49.589+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:49.666 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:49.666 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:49.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:49.719 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:49.717+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:30:49.759 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-17T17:30:49.344429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129499699,"num_objects":109,"num_object_clones":56,"num_object_copies":743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":13,"num_read":292,"num_read_kb":50315,"num_write":617,"num_write_kb":87458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":81362944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1808,"ondisk_log_size":1808,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":215,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":775320,"kb_used_data":259636,"kb_used_omap":24,"kb_used_meta":515303,"kb_avail":1509174120,"statfs":{"total":1546188226560,"available":1545394298880,"internally_reserved":0,"allocated":265867264,"data_stored":382129527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":527670351},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18481152,"num_objects":16,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":18,"num_read_kb":1299,"num_write":20,"num_write_kb":8096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":65880064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.005519"},"pg_stats":[{"pgid":"3.19","version":"65'44","reported_seq":371,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-17T17:30:44.353479+0000","last_change":"2024-09-17T17:30:44.354410+0000","last_active":"2024-09-17T17:30:44.353479+0000","last_peered":"2024-09-17T17:30:44.353479+0000","last_clean":"2024-09-17T17:30:44.353479+0000","last_became_active":"2024-09-17T17:30:26.345748+0000","last_became_peered":"2024-09-17T17:30:26.345748+0000","last_unstale":"2024-09-17T17:30:44.353479+0000","last_undegraded":"2024-09-17T17:30:44.353479+0000","last_fullsized":"2024-09-17T17:30:44.353479+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":5,"last_scrub":"65'44","last_scrub_stamp":"2024-09-17T17:30:27.911326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.911326+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:43.613031+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.004398608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"3.18","version":"76'29","reported_seq":399,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345277+0000","last_change":"2024-09-17T17:30:47.344007+0000","last_active":"2024-09-17T17:30:48.345277+0000","last_peered":"2024-09-17T17:30:48.345277+0000","last_clean":"2024-09-17T17:30:48.345277+0000","last_became_active":"2024-09-17T17:30:27.354929+0000","last_became_peered":"2024-09-17T17:30:27.354929+0000","last_unstale":"2024-09-17T17:30:48.345277+0000","last_undegraded":"2024-09-17T17:30:48.345277+0000","last_fullsized":"2024-09-17T17:30:48.345277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":5,"last_scrub":"76'29","last_scrub_stamp":"2024-09-17T17:30:28.377344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.377344+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:40.170345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013197230000000001,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":625,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.17","version":"40'6","reported_seq":333,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380416+0000","last_change":"2024-09-17T17:30:47.352062+0000","last_active":"2024-09-17T17:30:48.380416+0000","last_peered":"2024-09-17T17:30:48.380416+0000","last_clean":"2024-09-17T17:30:48.380416+0000","last_became_active":"2024-09-17T17:30:19.460414+0000","last_became_peered":"2024-09-17T17:30:19.460414+0000","last_unstale":"2024-09-17T17:30:48.380416+0000","last_undegraded":"2024-09-17T17:30:48.380416+0000","last_fullsized":"2024-09-17T17:30:48.380416+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:25.845481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.845481+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:36.124132+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0038344030000000001,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.16","version":"97'167","reported_seq":573,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.387831+0000","last_change":"2024-09-17T17:30:47.377754+0000","last_active":"2024-09-17T17:30:48.387831+0000","last_peered":"2024-09-17T17:30:48.387831+0000","last_clean":"2024-09-17T17:30:48.387831+0000","last_became_active":"2024-09-17T17:30:47.363797+0000","last_became_peered":"2024-09-17T17:30:47.363797+0000","last_unstale":"2024-09-17T17:30:48.387831+0000","last_undegraded":"2024-09-17T17:30:48.387831+0000","last_fullsized":"2024-09-17T17:30:48.387831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'146","last_scrub_stamp":"2024-09-17T17:30:36.055244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.055244+0000","objects_scrubbed":7,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:45.325510+0000","scrub_duration":133,"objects_trimmed":1,"snaptrim_duration":0.012742017,"stat_sum":{"num_bytes":13795328,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.15","version":"95'30","reported_seq":439,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367683+0000","last_change":"2024-09-17T17:30:47.358093+0000","last_active":"2024-09-17T17:30:48.367683+0000","last_peered":"2024-09-17T17:30:48.367683+0000","last_clean":"2024-09-17T17:30:48.367683+0000","last_became_active":"2024-09-17T17:30:46.350418+0000","last_became_peered":"2024-09-17T17:30:46.350418+0000","last_unstale":"2024-09-17T17:30:48.367683+0000","last_undegraded":"2024-09-17T17:30:48.367683+0000","last_fullsized":"2024-09-17T17:30:48.367683+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'20","last_scrub_stamp":"2024-09-17T17:30:27.352233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.352233+0000","objects_scrubbed":3,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:39.769790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0058574020000000003,"stat_sum":{"num_bytes":1736704,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":10,"num_read_kb":1250,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1736704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,2,7,11,8],"acting":[15,13,5,2,7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.14","version":"62'42","reported_seq":401,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T17:30:46.356059+0000","last_change":"2024-09-17T17:30:46.357309+0000","last_active":"2024-09-17T17:30:46.356059+0000","last_peered":"2024-09-17T17:30:46.356059+0000","last_clean":"2024-09-17T17:30:46.356059+0000","last_became_active":"2024-09-17T17:30:46.350731+0000","last_became_peered":"2024-09-17T17:30:46.350731+0000","last_unstale":"2024-09-17T17:30:46.356059+0000","last_undegraded":"2024-09-17T17:30:46.356059+0000","last_fullsized":"2024-09-17T17:30:46.356059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"62'42","last_scrub_stamp":"2024-09-17T17:30:26.686769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.686769+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.408891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050883450000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,15,9,2,6,10,13],"acting":[1,15,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367494+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:48.367494+0000","last_peered":"2024-09-17T17:30:48.367494+0000","last_clean":"2024-09-17T17:30:48.367494+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:48.367494+0000","last_undegraded":"2024-09-17T17:30:48.367494+0000","last_fullsized":"2024-09-17T17:30:48.367494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.115044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"97'139","reported_seq":518,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.376631+0000","last_change":"2024-09-17T17:30:47.377812+0000","last_active":"2024-09-17T17:30:48.376631+0000","last_peered":"2024-09-17T17:30:48.376631+0000","last_clean":"2024-09-17T17:30:48.376631+0000","last_became_active":"2024-09-17T17:30:47.362331+0000","last_became_peered":"2024-09-17T17:30:47.362331+0000","last_unstale":"2024-09-17T17:30:48.376631+0000","last_undegraded":"2024-09-17T17:30:48.376631+0000","last_fullsized":"2024-09-17T17:30:48.376631+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'127","last_scrub_stamp":"2024-09-17T17:30:27.836715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.836715+0000","objects_scrubbed":5,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:37.790365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.013564972,"stat_sum":{"num_bytes":3817472,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":6,"num_read_kb":669,"num_write":4,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367338+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:48.367338+0000","last_peered":"2024-09-17T17:30:48.367338+0000","last_clean":"2024-09-17T17:30:48.367338+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:48.367338+0000","last_undegraded":"2024-09-17T17:30:48.367338+0000","last_fullsized":"2024-09-17T17:30:48.367338+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:31.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":326,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380699+0000","last_change":"2024-09-17T17:30:47.352513+0000","last_active":"2024-09-17T17:30:48.380699+0000","last_peered":"2024-09-17T17:30:48.380699+0000","last_clean":"2024-09-17T17:30:48.380699+0000","last_became_active":"2024-09-17T17:30:19.457663+0000","last_became_peered":"2024-09-17T17:30:19.457663+0000","last_unstale":"2024-09-17T17:30:48.380699+0000","last_undegraded":"2024-09-17T17:30:48.380699+0000","last_fullsized":"2024-09-17T17:30:48.380699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:24.345150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.345150+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:45.462162+0000","scrub_duration":541,"objects_trimmed":0,"snaptrim_duration":0.0035564590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-17T17:30:44.349397+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:44.349397+0000","last_peered":"2024-09-17T17:30:44.349397+0000","last_clean":"2024-09-17T17:30:44.349397+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:44.349397+0000","last_undegraded":"2024-09-17T17:30:44.349397+0000","last_fullsized":"2024-09-17T17:30:44.349397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"95'60","reported_seq":444,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T17:30:46.374389+0000","last_change":"2024-09-17T17:30:46.375076+0000","last_active":"2024-09-17T17:30:46.374389+0000","last_peered":"2024-09-17T17:30:46.374389+0000","last_clean":"2024-09-17T17:30:46.374389+0000","last_became_active":"2024-09-17T17:30:46.352101+0000","last_became_peered":"2024-09-17T17:30:46.352101+0000","last_unstale":"2024-09-17T17:30:46.374389+0000","last_undegraded":"2024-09-17T17:30:46.374389+0000","last_fullsized":"2024-09-17T17:30:46.374389+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'48","last_scrub_stamp":"2024-09-17T17:30:27.676614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.676614+0000","objects_scrubbed":3,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:37.184044+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.021851948,"stat_sum":{"num_bytes":6848512,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13172736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,15,9,2,6,10,13],"acting":[1,15,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T17:30:46.344958+0000","last_change":"2024-09-17T17:29:34.506274+0000","last_active":"2024-09-17T17:30:46.344958+0000","last_peered":"2024-09-17T17:30:46.344958+0000","last_clean":"2024-09-17T17:30:46.344958+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:30:46.344958+0000","last_undegraded":"2024-09-17T17:30:46.344958+0000","last_fullsized":"2024-09-17T17:30:46.344958+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'27","reported_seq":416,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367929+0000","last_change":"2024-09-17T17:30:47.352462+0000","last_active":"2024-09-17T17:30:48.367929+0000","last_peered":"2024-09-17T17:30:48.367929+0000","last_clean":"2024-09-17T17:30:48.367929+0000","last_became_active":"2024-09-17T17:30:46.350505+0000","last_became_peered":"2024-09-17T17:30:46.350505+0000","last_unstale":"2024-09-17T17:30:48.367929+0000","last_undegraded":"2024-09-17T17:30:48.367929+0000","last_fullsized":"2024-09-17T17:30:48.367929+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'27","last_scrub_stamp":"2024-09-17T17:30:28.947105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.947105+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.988637+0000","scrub_duration":590,"objects_trimmed":0,"snaptrim_duration":0.00057592799999999997,"stat_sum":{"num_bytes":2113536,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,2,7,11,8],"acting":[15,13,5,2,7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.3","version":"97'83","reported_seq":463,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.370063+0000","last_change":"2024-09-17T17:30:47.353604+0000","last_active":"2024-09-17T17:30:48.370063+0000","last_peered":"2024-09-17T17:30:48.370063+0000","last_clean":"2024-09-17T17:30:48.370063+0000","last_became_active":"2024-09-17T17:30:26.345813+0000","last_became_peered":"2024-09-17T17:30:26.345813+0000","last_unstale":"2024-09-17T17:30:48.370063+0000","last_undegraded":"2024-09-17T17:30:48.370063+0000","last_fullsized":"2024-09-17T17:30:48.370063+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'58","last_scrub_stamp":"2024-09-17T17:30:28.942498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.942498+0000","objects_scrubbed":5,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:42.130059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001067045,"stat_sum":{"num_bytes":7880704,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":2592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.2","version":"28'2","reported_seq":159,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364427+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:48.364427+0000","last_peered":"2024-09-17T17:30:48.364427+0000","last_clean":"2024-09-17T17:30:48.364427+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:48.364427+0000","last_undegraded":"2024-09-17T17:30:48.364427+0000","last_fullsized":"2024-09-17T17:30:48.364427+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"93'84","reported_seq":440,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-17T17:30:44.368871+0000","last_change":"2024-09-17T17:30:44.369644+0000","last_active":"2024-09-17T17:30:44.368871+0000","last_peered":"2024-09-17T17:30:44.368871+0000","last_clean":"2024-09-17T17:30:44.368871+0000","last_became_active":"2024-09-17T17:30:27.343750+0000","last_became_peered":"2024-09-17T17:30:27.343750+0000","last_unstale":"2024-09-17T17:30:44.368871+0000","last_undegraded":"2024-09-17T17:30:44.368871+0000","last_fullsized":"2024-09-17T17:30:44.368871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'62","last_scrub_stamp":"2024-09-17T17:30:28.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.961363+0000","objects_scrubbed":3,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:47.029537+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.020697565000000001,"stat_sum":{"num_bytes":4014080,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":6919,"num_write":4,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.371808+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:30:48.371808+0000","last_peered":"2024-09-17T17:30:48.371808+0000","last_clean":"2024-09-17T17:30:48.371808+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:48.371808+0000","last_undegraded":"2024-09-17T17:30:48.371808+0000","last_fullsized":"2024-09-17T17:30:48.371808+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"96'47","reported_seq":413,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380032+0000","last_change":"2024-09-17T17:30:47.363646+0000","last_active":"2024-09-17T17:30:48.380032+0000","last_peered":"2024-09-17T17:30:48.380032+0000","last_clean":"2024-09-17T17:30:48.380032+0000","last_became_active":"2024-09-17T17:30:26.354825+0000","last_became_peered":"2024-09-17T17:30:26.354825+0000","last_unstale":"2024-09-17T17:30:48.380032+0000","last_undegraded":"2024-09-17T17:30:48.380032+0000","last_fullsized":"2024-09-17T17:30:48.380032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:28.852921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.852921+0000","objects_scrubbed":2,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.724409+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.018472654000000002,"stat_sum":{"num_bytes":3211264,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":8117,"num_write":6,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364635+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:30:48.364635+0000","last_peered":"2024-09-17T17:30:48.364635+0000","last_clean":"2024-09-17T17:30:48.364635+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:48.364635+0000","last_undegraded":"2024-09-17T17:30:48.364635+0000","last_fullsized":"2024-09-17T17:30:48.364635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"63'42","reported_seq":378,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.372087+0000","last_change":"2024-09-17T17:30:47.352662+0000","last_active":"2024-09-17T17:30:48.372087+0000","last_peered":"2024-09-17T17:30:48.372087+0000","last_clean":"2024-09-17T17:30:48.372087+0000","last_became_active":"2024-09-17T17:30:46.353966+0000","last_became_peered":"2024-09-17T17:30:46.353966+0000","last_unstale":"2024-09-17T17:30:48.372087+0000","last_undegraded":"2024-09-17T17:30:48.372087+0000","last_fullsized":"2024-09-17T17:30:48.372087+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'42","last_scrub_stamp":"2024-09-17T17:30:25.590903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.590903+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:36.105662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029960960000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0,3,4,15,10,11],"acting":[14,0,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.3","version":"26'1","reported_seq":166,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.371993+0000","last_change":"2024-09-17T17:30:38.649965+0000","last_active":"2024-09-17T17:30:48.371993+0000","last_peered":"2024-09-17T17:30:48.371993+0000","last_clean":"2024-09-17T17:30:48.371993+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:48.371993+0000","last_undegraded":"2024-09-17T17:30:48.371993+0000","last_fullsized":"2024-09-17T17:30:48.371993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:30:38.649873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.649873+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.468836+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345508+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:30:48.345508+0000","last_peered":"2024-09-17T17:30:48.345508+0000","last_clean":"2024-09-17T17:30:48.345508+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:48.345508+0000","last_undegraded":"2024-09-17T17:30:48.345508+0000","last_fullsized":"2024-09-17T17:30:48.345508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"75'29","reported_seq":396,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345464+0000","last_change":"2024-09-17T17:30:47.344306+0000","last_active":"2024-09-17T17:30:48.345464+0000","last_peered":"2024-09-17T17:30:48.345464+0000","last_clean":"2024-09-17T17:30:48.345464+0000","last_became_active":"2024-09-17T17:30:26.348354+0000","last_became_peered":"2024-09-17T17:30:26.348354+0000","last_unstale":"2024-09-17T17:30:48.345464+0000","last_undegraded":"2024-09-17T17:30:48.345464+0000","last_fullsized":"2024-09-17T17:30:48.345464+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'29","last_scrub_stamp":"2024-09-17T17:30:27.407808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.407808+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:53.792072+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.001223395,"stat_sum":{"num_bytes":344064,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.9","version":"92'67","reported_seq":438,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-17T17:30:44.349594+0000","last_change":"2024-09-17T17:30:44.350162+0000","last_active":"2024-09-17T17:30:44.349594+0000","last_peered":"2024-09-17T17:30:44.349594+0000","last_clean":"2024-09-17T17:30:44.349594+0000","last_became_active":"2024-09-17T17:30:28.394697+0000","last_became_peered":"2024-09-17T17:30:28.394697+0000","last_unstale":"2024-09-17T17:30:44.349594+0000","last_undegraded":"2024-09-17T17:30:44.349594+0000","last_fullsized":"2024-09-17T17:30:44.349594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'61","last_scrub_stamp":"2024-09-17T17:30:36.024221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.024221+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:00.297315+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000817708,"stat_sum":{"num_bytes":7012352,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"3.a","version":"92'136","reported_seq":451,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364231+0000","last_change":"2024-09-17T17:30:47.348842+0000","last_active":"2024-09-17T17:30:48.364231+0000","last_peered":"2024-09-17T17:30:48.364231+0000","last_clean":"2024-09-17T17:30:48.364231+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:48.364231+0000","last_undegraded":"2024-09-17T17:30:48.364231+0000","last_fullsized":"2024-09-17T17:30:48.364231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:49.643669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001344788,"stat_sum":{"num_bytes":14680064,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":52,"num_read_kb":15272,"num_write":108,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.b","version":"96'22","reported_seq":361,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380199+0000","last_change":"2024-09-17T17:30:47.372788+0000","last_active":"2024-09-17T17:30:48.380199+0000","last_peered":"2024-09-17T17:30:48.380199+0000","last_clean":"2024-09-17T17:30:48.380199+0000","last_became_active":"2024-09-17T17:30:47.358299+0000","last_became_peered":"2024-09-17T17:30:47.358299+0000","last_unstale":"2024-09-17T17:30:48.380199+0000","last_undegraded":"2024-09-17T17:30:48.380199+0000","last_fullsized":"2024-09-17T17:30:48.380199+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:08.923689+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013131362000000001,"stat_sum":{"num_bytes":2310144,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":6719,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4538368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,10,6,8,7,14],"acting":[13,3,10,6,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.c","version":"97'156","reported_seq":463,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.379310+0000","last_change":"2024-09-17T17:30:47.347763+0000","last_active":"2024-09-17T17:30:48.379310+0000","last_peered":"2024-09-17T17:30:48.379310+0000","last_clean":"2024-09-17T17:30:48.379310+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:48.379310+0000","last_undegraded":"2024-09-17T17:30:48.379310+0000","last_fullsized":"2024-09-17T17:30:48.379310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002112245,"stat_sum":{"num_bytes":17776640,"num_objects":13,"num_object_clones":8,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":45,"num_read_kb":3633,"num_write":101,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.d","version":"96'97","reported_seq":389,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T17:30:47.353500+0000","last_change":"2024-09-17T17:30:47.354659+0000","last_active":"2024-09-17T17:30:47.353500+0000","last_peered":"2024-09-17T17:30:47.353500+0000","last_clean":"2024-09-17T17:30:47.353500+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:30:47.353500+0000","last_undegraded":"2024-09-17T17:30:47.353500+0000","last_fullsized":"2024-09-17T17:30:47.353500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:08.625403+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0094019519999999999,"stat_sum":{"num_bytes":9322496,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":15,"num_read_kb":3611,"num_write":58,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.e","version":"93'60","reported_seq":351,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-17T17:30:44.496056+0000","last_change":"2024-09-17T17:30:44.350227+0000","last_active":"2024-09-17T17:30:44.496056+0000","last_peered":"2024-09-17T17:30:44.496056+0000","last_clean":"2024-09-17T17:30:44.496056+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:44.496056+0000","last_undegraded":"2024-09-17T17:30:44.496056+0000","last_fullsized":"2024-09-17T17:30:44.496056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:54.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056916699999999998,"stat_sum":{"num_bytes":6520832,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3129,"num_write":51,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"3.f","version":"96'94","reported_seq":494,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364480+0000","last_change":"2024-09-17T17:30:47.372219+0000","last_active":"2024-09-17T17:30:48.364480+0000","last_peered":"2024-09-17T17:30:48.364480+0000","last_clean":"2024-09-17T17:30:48.364480+0000","last_became_active":"2024-09-17T17:30:47.356042+0000","last_became_peered":"2024-09-17T17:30:47.356042+0000","last_unstale":"2024-09-17T17:30:48.364480+0000","last_undegraded":"2024-09-17T17:30:48.364480+0000","last_fullsized":"2024-09-17T17:30:48.364480+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.145356+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01447214,"stat_sum":{"num_bytes":15581184,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":74,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":41320448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,4,14,2,3,0],"acting":[11,8,4,14,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.10","version":"77'53","reported_seq":159,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345149+0000","last_change":"2024-09-17T17:30:47.356852+0000","last_active":"2024-09-17T17:30:48.345149+0000","last_peered":"2024-09-17T17:30:48.345149+0000","last_clean":"2024-09-17T17:30:48.345149+0000","last_became_active":"2024-09-17T17:30:47.354125+0000","last_became_peered":"2024-09-17T17:30:47.354125+0000","last_unstale":"2024-09-17T17:30:48.345149+0000","last_undegraded":"2024-09-17T17:30:48.345149+0000","last_fullsized":"2024-09-17T17:30:48.345149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'52","last_scrub_stamp":"2024-09-17T17:30:26.907313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.907313+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:32.936798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001721717,"stat_sum":{"num_bytes":2342912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7,14,5,9,11],"acting":[6,0,7,14,5,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.11","version":"60'25","reported_seq":84,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-17T17:30:44.344529+0000","last_change":"2024-09-17T17:30:44.344885+0000","last_active":"2024-09-17T17:30:44.344529+0000","last_peered":"2024-09-17T17:30:44.344529+0000","last_clean":"2024-09-17T17:30:44.344529+0000","last_became_active":"2024-09-17T17:30:36.353414+0000","last_became_peered":"2024-09-17T17:30:36.353414+0000","last_unstale":"2024-09-17T17:30:44.344529+0000","last_undegraded":"2024-09-17T17:30:44.344529+0000","last_fullsized":"2024-09-17T17:30:44.344529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:37.309789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:37.309789+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:57.352744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063069199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,10,5,14,2],"acting":[4,11,15,10,5,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"3.12","version":"93'84","reported_seq":471,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.371976+0000","last_change":"2024-09-17T17:30:47.350338+0000","last_active":"2024-09-17T17:30:48.371976+0000","last_peered":"2024-09-17T17:30:48.371976+0000","last_clean":"2024-09-17T17:30:48.371976+0000","last_became_active":"2024-09-17T17:30:39.350802+0000","last_became_peered":"2024-09-17T17:30:39.350802+0000","last_unstale":"2024-09-17T17:30:48.371976+0000","last_undegraded":"2024-09-17T17:30:48.371976+0000","last_fullsized":"2024-09-17T17:30:48.371976+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'48","last_scrub_stamp":"2024-09-17T17:30:24.093396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.093396+0000","objects_scrubbed":3,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.256082+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0.0010622699999999999,"stat_sum":{"num_bytes":3047424,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":9,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6160384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,7,5,4,11,3,15],"acting":[14,7,5,4,11,3,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.13","version":"83'64","reported_seq":465,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364524+0000","last_change":"2024-09-17T17:30:47.348278+0000","last_active":"2024-09-17T17:30:48.364524+0000","last_peered":"2024-09-17T17:30:48.364524+0000","last_clean":"2024-09-17T17:30:48.364524+0000","last_became_active":"2024-09-17T17:30:46.350583+0000","last_became_peered":"2024-09-17T17:30:46.350583+0000","last_unstale":"2024-09-17T17:30:48.364524+0000","last_undegraded":"2024-09-17T17:30:48.364524+0000","last_fullsized":"2024-09-17T17:30:48.364524+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'52","last_scrub_stamp":"2024-09-17T17:30:27.908152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.908152+0000","objects_scrubbed":2,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:47.701212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0022866409999999998,"stat_sum":{"num_bytes":4063232,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7634944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,6,5,4,13],"acting":[11,7,2,6,5,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":128909312,"num_objects":105,"num_object_clones":56,"num_object_copies":735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":13,"num_read":206,"num_read_kb":50242,"num_write":470,"num_write_kb":84784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":81362944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245641216,"data_stored":372830208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1693,"ondisk_log_size":1693,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247826,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52928,"kb_used_data":17576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94318912,"statfs":{"total":96636764160,"available":96582565888,"internally_reserved":0,"allocated":17997824,"data_stored":34123871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":19612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":20082688,"data_stored":29804015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39892,"kb_used_data":8700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331948,"statfs":{"total":96636764160,"available":96595914752,"internally_reserved":0,"allocated":8908800,"data_stored":9829461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313235,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54992,"kb_used_data":19640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94316848,"statfs":{"total":96636764160,"available":96580452352,"internally_reserved":0,"allocated":20111360,"data_stored":32918868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44860,"kb_used_data":13672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326980,"statfs":{"total":96636764160,"available":96590827520,"internally_reserved":0,"allocated":14000128,"data_stored":21587439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48264,"kb_used_data":17072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323576,"statfs":{"total":96636764160,"available":96587341824,"internally_reserved":0,"allocated":17481728,"data_stored":23653716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280532,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":19164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":19623936,"data_stored":23320591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46680,"kb_used_data":15488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325160,"statfs":{"total":96636764160,"available":96588963840,"internally_reserved":0,"allocated":15859712,"data_stored":22726127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49544,"kb_used_data":14188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322296,"statfs":{"total":96636764160,"available":96586031104,"internally_reserved":0,"allocated":14528512,"data_stored":18896367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280532,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48300,"kb_used_data":12952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323540,"statfs":{"total":96636764160,"available":96587304960,"internally_reserved":0,"allocated":13262848,"data_stored":21966355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280531,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42888,"kb_used_data":11700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328952,"statfs":{"total":96636764160,"available":96592846848,"internally_reserved":0,"allocated":11980800,"data_stored":17966575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,7,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46152,"kb_used_data":14964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325688,"statfs":{"total":96636764160,"available":96589504512,"internally_reserved":0,"allocated":15323136,"data_stored":20207087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47212,"kb_used_data":20180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94324628,"statfs":{"total":96636764160,"available":96588419072,"internally_reserved":0,"allocated":20664320,"data_stored":31000047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53292,"kb_used_data":22104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318548,"statfs":{"total":96636764160,"available":96582193152,"internally_reserved":0,"allocated":22634496,"data_stored":26220034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247827,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50400,"kb_used_data":15048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321440,"statfs":{"total":96636764160,"available":96585154560,"internally_reserved":0,"allocated":15409152,"data_stored":26300511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48764,"kb_used_data":17576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323076,"statfs":{"total":96636764160,"available":96586829824,"internally_reserved":0,"allocated":17997824,"data_stored":21608463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17825792,"data_stored":22319104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18714624,"data_stored":32104448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19464192,"data_stored":31150080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21430272,"data_stored":26370048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13488128,"data_stored":24739840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13185024,"data_stored":18141184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16199680,"data_stored":20606976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16064512,"data_stored":21340160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10190848,"data_stored":11681792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":11829248,"data_stored":21131264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10633216,"data_stored":17764352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":18739200,"data_stored":29048832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":16650240,"data_stored":33341440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":14516224,"data_stored":21970944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":12734464,"data_stored":21114880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":13975552,"data_stored":20004864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:30:49.762 INFO:tasks.thrashosds.thrasher:Setting 3.5 to [10, 0, 5, 2, 7, 14, 3] 2024-09-17T17:30:49.762 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.5', '10', '0', '5', '2', '7', '14', '3'] 2024-09-17T17:30:49.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.5 10 0 5 2 7 14 3 2024-09-17T17:30:49.784 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:49.784 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:49.792 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:49.790+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:49.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:49.893 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:49.891+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:49.947 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:49.947 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:49.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:49.994 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:49.993+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:50.063 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.063 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.072 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:50.096 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:50.094+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:50.179 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.179 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:50.197 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:50.195+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:50.295 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.295 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.298 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:50.296+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:50.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:50.399 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:50.397+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:50.412 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.412 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:50.500 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:50.498+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:30:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.5 pg_upmap mapping to [10,0,5,2,7,14,3] 2024-09-17T17:30:50.535 INFO:tasks.rados.rados.0.smithi102.stdout:380: done (1 left) 2024-09-17T17:30:50.535 INFO:tasks.rados.rados.0.smithi102.stdout:381: done (0 left) 2024-09-17T17:30:50.535 INFO:tasks.rados.rados.0.smithi102.stdout:382: snap_remove snap 20 2024-09-17T17:30:50.572 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.572 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:50.622 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:50.620+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:50.702 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:50.700+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:50.729 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.729 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:50.803 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:50.802+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:50.845 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.845 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:50.904 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:50.903+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:50.961 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:50.962 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:50.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:51.015 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:51.013+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:51.091 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:51.091 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:51.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:51.107 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:51.105+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:51.208 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:51.206+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:51.248 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:51.248 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:51.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:51.308 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:51.306+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:51.364 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:51.364 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:51.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:51.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:51.409 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:51.407+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:51.512 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:51.508+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:51.521 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:51.521 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:51.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:51.539 INFO:tasks.rados.rados.0.smithi102.stdout:382: done (0 left) 2024-09-17T17:30:51.539 INFO:tasks.rados.rados.0.smithi102.stdout:383: rmattr oid 48 current snap is 33 2024-09-17T17:30:51.539 INFO:tasks.rados.rados.0.smithi102.stdout:383: done (0 left) 2024-09-17T17:30:51.539 INFO:tasks.rados.rados.0.smithi102.stdout:384: rollback oid 10 current snap is 33 2024-09-17T17:30:51.539 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 10 to 31 2024-09-17T17:30:51.539 INFO:tasks.rados.rados.0.smithi102.stdout:385: delete oid 37 current snap is 33 2024-09-17T17:30:51.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:51.548 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:51.554 INFO:tasks.rados.rados.0.smithi102.stdout:385: done (1 left) 2024-09-17T17:30:51.554 INFO:tasks.rados.rados.0.smithi102.stdout:386: delete oid 11 current snap is 33 2024-09-17T17:30:51.567 INFO:tasks.rados.rados.0.smithi102.stdout:384: finishing rollback tid 0 to smithi10238019-10 2024-09-17T17:30:51.567 INFO:tasks.rados.rados.0.smithi102.stdout:384: finishing rollback tid 1 to smithi10238019-10 2024-09-17T17:30:51.567 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 10 v 146 (ObjNum 33 snap 0 seq_num 3482148901) dirty dne 2024-09-17T17:30:51.576 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:51.576 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:51.597 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:51.611 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:51.609+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:51.619 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:51.620 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:51.640 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:51.650 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:51.650 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:51.661 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:51.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:51.662 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:51.684 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:51.706 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:51.707 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:51.711 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:51.709+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:51.727 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:51.727 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:51.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:51.749 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:51.754 INFO:tasks.rados.rados.0.smithi102.stdout:384: done (1 left) 2024-09-17T17:30:51.754 INFO:tasks.rados.rados.0.smithi102.stdout:386: done (0 left) 2024-09-17T17:30:51.754 INFO:tasks.rados.rados.0.smithi102.stdout:387: read oid 44 snap -1 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:387: expect deleted 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:388: read oid 35 snap 14 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:388: expect (ObjNum 101 snap 13 seq_num 101) 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:389: copy_from oid 40 from oid 27 current snap is 33 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:390: read oid 15 snap 21 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:390: expect (ObjNum 122 snap 19 seq_num 122) 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:391: copy_from oid 46 from oid 45 current snap is 33 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:392: copy_from oid 26 from oid 4 current snap is 33 2024-09-17T17:30:51.755 INFO:tasks.rados.rados.0.smithi102.stdout:393: delete oid 38 current snap is 33 2024-09-17T17:30:51.760 INFO:tasks.rados.rados.0.smithi102.stdout:387: done (6 left) 2024-09-17T17:30:51.760 INFO:tasks.rados.rados.0.smithi102.stdout:393: done (5 left) 2024-09-17T17:30:51.760 INFO:tasks.rados.rados.0.smithi102.stdout:394: rmattr oid 9 current snap is 33 2024-09-17T17:30:51.760 INFO:tasks.rados.rados.0.smithi102.stdout:394: done (5 left) 2024-09-17T17:30:51.760 INFO:tasks.rados.rados.0.smithi102.stdout:395: delete oid 13 current snap is 33 2024-09-17T17:30:51.766 INFO:tasks.rados.rados.0.smithi102.stdout:395: done (5 left) 2024-09-17T17:30:51.767 INFO:tasks.rados.rados.0.smithi102.stdout:396: rollback oid 8 current snap is 33 2024-09-17T17:30:51.767 INFO:tasks.rados.rados.0.smithi102.stdout:rollback oid 8 to 30 2024-09-17T17:30:51.767 INFO:tasks.rados.rados.0.smithi102.stdout:397: rmattr oid 14 current snap is 33 2024-09-17T17:30:51.767 INFO:tasks.rados.rados.0.smithi102.stdout:397: done (6 left) 2024-09-17T17:30:51.767 INFO:tasks.rados.rados.0.smithi102.stdout:398: delete oid 5 current snap is 33 2024-09-17T17:30:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:51.775 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:51.776 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 62.781169220999999, 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:30:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:30:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:30:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:30:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:30:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:30:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:30:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:30:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:30:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:30:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:30:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.047096181000001, 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:30:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:30:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:30:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:30:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:30:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:30:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.079068309, 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.078826198999998, 2024-09-17T17:30:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:30:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:30:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:30:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:30:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/32 ECSubWrite(tid=105, reqid=client.4566.0:425, at_version=52'39, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 48.383292060000002, 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027457129999999999, 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.389643+0000", 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2200000000000004e-07 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390408+0000", 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00076470399999999995 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390444+0000", 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5933000000000003e-05 2024-09-17T17:30:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390453+0000", 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5120000000000008e-06 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390525+0000", 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2849000000000007e-05 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.390537+0000", 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1999e-05 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392350+0000", 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018122660000000001 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:03.392389+0000", 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8828000000000002e-05 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.388334381999996, 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029370899999999998, 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384601+0000", 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8e-07 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385325+0000", 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072435200000000005 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385406+0000", 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0667999999999994e-05 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385415+0000", 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385509+0000", 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4137999999999999e-05 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385519+0000", 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0208e-05 2024-09-17T17:30:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387496+0000", 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001976387 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387538+0000", 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1996999999999998e-05 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.387606220000002, 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029662, 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385362+0000", 2024-09-17T17:30:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2403999999999997e-05 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385471+0000", 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109485 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385479+0000", 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6750000000000002e-06 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386169+0000", 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068989099999999999 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386179+0000", 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0278000000000001e-05 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388245+0000", 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020665150000000001 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388295+0000", 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9369000000000001e-05 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 32.33385912, 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029678690000000001, 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1899999999999998e-07 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439101+0000", 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4655e-05 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439183+0000", 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2472000000000006e-05 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439193+0000", 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0063e-05 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439257+0000", 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3775000000000005e-05 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439270+0000", 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2588e-05 2024-09-17T17:30:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441995+0000", 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002725505 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442044+0000", 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8392e-05 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 70/67 ECSubWrite(tid=155, reqid=osd.6.0:155, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.427766449, 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00308939, 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345169+0000", 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345192+0000", 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3241e-05 2024-09-17T17:30:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345239+0000", 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6607999999999998e-05 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345247+0000", 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5339999999999999e-06 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345711+0000", 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00046389400000000001 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345720+0000", 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0710000000000001e-06 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348210+0000", 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024892099999999999 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348258+0000", 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8208000000000003e-05 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=199, reqid=osd.11.0:199, at_version=70'104, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.427472441999999, 2024-09-17T17:30:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002830782, 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345463+0000", 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345486+0000", 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3453999999999999e-05 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345559+0000", 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2760000000000001e-05 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345586+0000", 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7670000000000001e-05 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345650+0000", 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3139999999999995e-05 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345661+0000", 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1342000000000001e-05 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348252+0000", 2024-09-17T17:30:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025906829999999999 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348293+0000", 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.155e-05 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=200, reqid=osd.11.0:200, at_version=70'106, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 30.426891007999998, 2024-09-17T17:30:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027884030000000001, 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346045+0000", 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346081+0000", 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5942999999999998e-05 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346125+0000", 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3949999999999998e-05 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346134+0000", 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5019999999999998e-06 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346909+0000", 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077504099999999997 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346921+0000", 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1598e-05 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348789+0000", 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001868268 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348832+0000", 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3059000000000002e-05 2024-09-17T17:30:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.429533961000001, 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003502827, 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343433+0000", 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2026999999999997e-05 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343478+0000", 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4561999999999998e-05 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343489+0000", 2024-09-17T17:30:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343600+0000", 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000110818 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343617+0000", 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6824999999999999e-05 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346851+0000", 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032341750000000002 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346904+0000", 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2850999999999999e-05 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 91/32 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=91'144, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.4280940379999993, 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004955597, 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5300000000000004e-07 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344871+0000", 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0083e-05 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344995+0000", 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012343400000000001 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345025+0000", 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0437999999999999e-05 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345098+0000", 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3114999999999994e-05 2024-09-17T17:30:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345110+0000", 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2004999999999999e-05 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349747+0000", 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046367179999999997 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349796+0000", 2024-09-17T17:30:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9251e-05 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 91/32 ECSubWrite(tid=101, reqid=osd.7.0:101, at_version=91'83, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.4274175390000003, 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0081096810000000005, 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7499999999999999e-07 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345541+0000", 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3065999999999999e-05 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345608+0000", 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7149000000000004e-05 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345617+0000", 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345692+0000", 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5053000000000001e-05 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345703+0000", 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1172000000000001e-05 2024-09-17T17:30:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353581+0000", 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0078780229999999996 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353627+0000", 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5759e-05 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 91/32 ECSubWrite(tid=102, reqid=osd.7.0:102, at_version=91'85, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.4273914689999998, 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0089760780000000002, 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345544+0000", 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.75e-07 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345564+0000", 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0519000000000001e-05 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348824+0000", 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032595050000000002 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348834+0000", 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0329000000000001e-05 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349653+0000", 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000818445 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349662+0000", 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5119999999999997e-06 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354468+0000", 2024-09-17T17:30:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004805363 2024-09-17T17:30:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354519+0000", 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1929999999999999e-05 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.4240687699999999, 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003290444, 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348867+0000", 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348892+0000", 2024-09-17T17:30:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5429e-05 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348942+0000", 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9731999999999998e-05 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348950+0000", 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3350000000000007e-06 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349267+0000", 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00031690899999999998 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349280+0000", 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2880999999999999e-05 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352124+0000", 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028441819999999998 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352157+0000", 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2177e-05 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 91/32 ECSubWrite(tid=75, reqid=client.4566.0:895, at_version=91'58, trim_to=0'0, roll_forward_to=83'57))", 2024-09-17T17:30:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:30:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.3895314889999995, 2024-09-17T17:30:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0040288700000000004, 2024-09-17T17:30:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:30:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383404+0000", 2024-09-17T17:30:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6399999999999998e-07 2024-09-17T17:30:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385486+0000", 2024-09-17T17:30:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020826339999999999 2024-09-17T17:30:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385523+0000", 2024-09-17T17:30:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6739999999999997e-05 2024-09-17T17:30:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385534+0000", 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0937e-05 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385626+0000", 2024-09-17T17:30:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2428000000000003e-05 2024-09-17T17:30:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385655+0000", 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8343e-05 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.387395+0000", 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001740157 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.387432+0000", 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7267000000000001e-05 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 92/32 ECSubWrite(tid=282, reqid=osd.11.0:282, at_version=92'148, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.4229592049999997, 2024-09-17T17:30:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029263319999999998, 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4999999999999999e-07 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350009+0000", 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2889999999999999e-05 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350288+0000", 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00027886300000000002 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350299+0000", 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1931e-05 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350372+0000", 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2140999999999999e-05 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350386+0000", 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4194e-05 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352851+0000", 2024-09-17T17:30:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024656330000000001 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352902+0000", 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0529999999999999e-05 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 92/32 ECSubWrite(tid=283, reqid=osd.11.0:283, at_version=92'150, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:30:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.4229230610000005, 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029327540000000001, 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4999999999999998e-07 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350038+0000", 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5956e-05 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350867+0000", 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00082833600000000005 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350881+0000", 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4865000000000001e-05 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350949+0000", 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7794999999999997e-05 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350960+0000", 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1083e-05 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352909+0000", 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019484750000000001 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352945+0000", 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5794000000000003e-05 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 3.3949663019999998, 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0037244769999999999, 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6499999999999999e-07 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378414+0000", 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044483400000000002 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378460+0000", 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5980999999999999e-05 2024-09-17T17:30:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378468+0000", 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3049999999999993e-06 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378582+0000", 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000114036 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378597+0000", 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4647000000000001e-05 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.381637+0000", 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003039698 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.381693+0000", 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6410999999999999e-05 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:51.892 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:51.893 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:51.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:51.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:51.897 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:51.897 INFO:tasks.rados.rados.0.smithi102.stdout:391: finishing copy_from racing read to smithi10238019-46 2024-09-17T17:30:51.897 INFO:tasks.rados.rados.0.smithi102.stdout:398: done (6 left) 2024-09-17T17:30:51.897 INFO:tasks.rados.rados.0.smithi102.stdout:append oid 9 current snap is 33 2024-09-17T17:30:51.897 INFO:tasks.rados.rados.0.smithi102.stdout:399: seq_num 169 ranges {819200=442368} 2024-09-17T17:30:51.897 INFO:tasks.rados.rados.0.smithi102.stdout:399: writing smithi10238019-9 from 819200 to 1261568 tid 1 2024-09-17T17:30:51.898 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 7 2024-09-17T17:30:51.898 INFO:tasks.rados.rados.0.smithi102.stdout:388: expect (ObjNum 101 snap 13 seq_num 101) 2024-09-17T17:30:51.898 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:51.898 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 2161091 2024-09-17T17:30:51.898 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:51.847+0000 7f99c7fff640 0 -- 172.21.15.7:0/518489638 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7f99c821f130 msgr2=0x7f99c8221520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:51.898 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:51.899 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:51.899 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:51.891+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:51.905 INFO:tasks.rados.rados.0.smithi102.stdout:390: expect (ObjNum 122 snap 19 seq_num 122) 2024-09-17T17:30:51.924 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 3362107 2024-09-17T17:30:51.927 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 0 2024-09-17T17:30:51.981 INFO:tasks.rados.rados.0.smithi102.stdout:seeking to 1945998 2024-09-17T17:30:51.994 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:51.992+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:396: finishing rollback tid 0 to smithi10238019-8 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:388: done (6 left) 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:390: done (5 left) 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 5 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:392: finishing copy_from racing read to smithi10238019-26 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:396: finishing rollback tid 1 to smithi10238019-8 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:396: finishing rollback tid 2 to smithi10238019-8 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 8 v 181 (ObjNum 99 snap 13 seq_num 99) dirty exists 2024-09-17T17:30:51.998 INFO:tasks.rados.rados.0.smithi102.stdout:391: finishing copy_from to smithi10238019-46 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 46 v 153 (ObjNum 142 snap 27 seq_num 142) dirty exists 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:391: done (4 left) 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:396: done (3 left) 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 3 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:392: finishing copy_from to smithi10238019-26 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 26 v 32 (ObjNum 53 snap 2 seq_num 53) dirty exists 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:389: finishing copy_from racing read to smithi10238019-40 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:399: finishing write tid 1 to smithi10238019-9 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:399: finishing write tid 2 to smithi10238019-9 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:392: done (2 left) 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout: waiting on 2 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:389: finishing copy_from to smithi10238019-40 2024-09-17T17:30:51.999 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 40 v 60 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:399: finishing write tid 3 to smithi10238019-9 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:399: oid 9 updating version 0 to 95 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:399: oid 9 version 95 is already newer than 94 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:update_object_version oid 9 v 95 (ObjNum 169 snap 33 seq_num 169) dirty exists 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:399: left oid 9 (ObjNum 169 snap 33 seq_num 169) 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:389: done (1 left) 2024-09-17T17:30:52.000 INFO:tasks.rados.rados.0.smithi102.stdout:399: done (0 left) 2024-09-17T17:30:52.001 INFO:tasks.rados.rados.0.smithi102.stderr:0 errors. 2024-09-17T17:30:52.001 INFO:tasks.rados.rados.0.smithi102.stderr: 2024-09-17T17:30:52.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:30:52.007 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:52.007 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:52.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:52.095 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:52.093+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:52.122 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:52.123 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:52.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:52.196 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:52.195+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:52.279 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.938042041000003, 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032487919999999999, 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339225+0000", 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6799999999999999e-07 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339248+0000", 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3654000000000001e-05 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339288+0000", 2024-09-17T17:30:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9734999999999998e-05 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339297+0000", 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339672+0000", 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037528099999999999 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339682+0000", 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0168e-05 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342436+0000", 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027540889999999999 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342473+0000", 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6331999999999997e-05 2024-09-17T17:30:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.937863942, 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030245340000000002, 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:30:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0100000000000001e-07 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339459+0000", 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6342999999999999e-05 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339692+0000", 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023318199999999999 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339701+0000", 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7469999999999994e-06 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339766+0000", 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5277999999999994e-05 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339776+0000", 2024-09-17T17:30:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8439999999999993e-06 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342401+0000", 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026253349999999999 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342426+0000", 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5503999999999998e-05 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=92, reqid=osd.12.0:92, at_version=56'40, trim_to=0'0, roll_forward_to=56'34))", 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.923354439000001, 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003074832, 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353912+0000", 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353937+0000", 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5015e-05 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354085+0000", 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014835900000000001 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354091+0000", 2024-09-17T17:30:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8220000000000003e-06 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356082+0000", 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019913650000000001 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356088+0000", 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1190000000000002e-06 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356953+0000", 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00086492399999999999 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356986+0000", 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2994999999999998e-05 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.904812941000003, 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030867720000000002, 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2800000000000001e-07 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372483+0000", 2024-09-17T17:30:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9967999999999998e-05 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372565+0000", 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2091999999999999e-05 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372574+0000", 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8869999999999998e-06 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372659+0000", 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5358999999999994e-05 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372674+0000", 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4827000000000001e-05 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375499+0000", 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028244870000000001 2024-09-17T17:30:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375540+0000", 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1023999999999999e-05 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 43.903562268000002, 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033844539999999998, 2024-09-17T17:30:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373730+0000", 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5689000000000001e-05 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373771+0000", 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1208000000000003e-05 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374141+0000", 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037011200000000002 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374273+0000", 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013144900000000001 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374283+0000", 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0147e-05 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377050+0000", 2024-09-17T17:30:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002767346 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377088+0000", 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7713000000000003e-05 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 32.836666022000003, 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003708654, 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5499999999999999e-07 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440633+0000", 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2728999999999998e-05 2024-09-17T17:30:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440769+0000", 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013610200000000001 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440779+0000", 2024-09-17T17:30:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9410000000000002e-06 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442220+0000", 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0014405760000000001 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442231+0000", 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1847e-05 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444268+0000", 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020363299999999998 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444309+0000", 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0774e-05 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 68/67 ECSubWrite(tid=122, reqid=client.4566.0:616, at_version=68'46, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 32.833192959000002, 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030735390000000001, 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444074+0000", 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1200000000000002e-07 2024-09-17T17:30:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444177+0000", 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010335700000000001 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444221+0000", 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4370999999999999e-05 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444232+0000", 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0178e-05 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444294+0000", 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2204000000000003e-05 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444303+0000", 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9630000000000004e-06 2024-09-17T17:30:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447110+0000", 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002806809 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447146+0000", 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6944999999999999e-05 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=157, to_read={3:00c2d3b1:::smithi10238019-29:head=0,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.461983589999999, 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30626450500000002, 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815283+0000", 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9299999999999998e-07 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815290+0000", 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8709999999999999e-06 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121092+0000", 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30580278100000002 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121102+0000", 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5170000000000005e-06 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121186+0000", 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4229999999999998e-05 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121547+0000", 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00036061300000000003 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=171, to_read={3:00c2d3b1:::smithi10238019-29:head=262144,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 27.153146097, 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.21319038800000001, 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:30:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.61e-07 2024-09-17T17:30:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124129+0000", 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5269999999999998e-06 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336658+0000", 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.212529417 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336671+0000", 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2658e-05 2024-09-17T17:30:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336735+0000", 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4203999999999997e-05 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.337310+0000", 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057492100000000005 2024-09-17T17:30:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 85/32 ECSubWrite(tid=83, reqid=osd.7.0:83, at_version=85'73, trim_to=0'0, roll_forward_to=83'71))", 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 15.930722597000001, 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028233020000000002, 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346544+0000", 2024-09-17T17:30:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3099999999999999e-07 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346574+0000", 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0852999999999998e-05 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346642+0000", 2024-09-17T17:30:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7183999999999999e-05 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346653+0000", 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1076e-05 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346728+0000", 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5459999999999999e-05 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346741+0000", 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2894999999999999e-05 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.349319+0000", 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00257771 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.349367+0000", 2024-09-17T17:30:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7793000000000001e-05 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.933823402, 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030163410000000001, 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2100000000000001e-07 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343467+0000", 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4273e-05 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343522+0000", 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5386999999999997e-05 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343534+0000", 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.198e-05 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343611+0000", 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6996999999999995e-05 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343623+0000", 2024-09-17T17:30:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2194999999999999e-05 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346410+0000", 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002786602 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346459+0000", 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8785999999999997e-05 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s2 90/74 ECSubWrite(tid=76, reqid=client.4566.0:887, at_version=90'58, trim_to=0'0, roll_forward_to=83'56))", 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.344904+0000", 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.932361658, 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0035865720000000001, 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.344904+0000", 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.344904+0000", 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.344905+0000", 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9200000000000002e-07 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345022+0000", 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00011770899999999999 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345095+0000", 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2934000000000003e-05 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345106+0000", 2024-09-17T17:30:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0342e-05 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345191+0000", 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5152000000000001e-05 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345202+0000", 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1195999999999999e-05 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.348449+0000", 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032475669999999998 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.348491+0000", 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.138e-05 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 91/32 ECSubWrite(tid=101, reqid=osd.7.0:101, at_version=91'83, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.931628946, 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004051476, 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3000000000000002e-07 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345666+0000", 2024-09-17T17:30:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8405999999999999e-05 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345755+0000", 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8998000000000004e-05 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345764+0000", 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.38e-06 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345859+0000", 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4679000000000006e-05 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345871+0000", 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2084e-05 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349647+0000", 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003776106 2024-09-17T17:30:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349688+0000", 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1492999999999997e-05 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 91/32 ECSubWrite(tid=102, reqid=osd.7.0:102, at_version=91'85, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.9315970940000007, 2024-09-17T17:30:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0061800359999999999, 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6699999999999999e-07 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345715+0000", 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5666e-05 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345800+0000", 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4781e-05 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345807+0000", 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9380000000000003e-06 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348723+0000", 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00291647 2024-09-17T17:30:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348733+0000", 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9459999999999993e-06 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351805+0000", 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003071823 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351849+0000", 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3945e-05 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.346604+0000", 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.9306620900000002, 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046338799999999999, 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346604+0000", 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346604+0000", 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346605+0000", 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.879999999999999e-07 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346630+0000", 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4813000000000001e-05 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346676+0000", 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6278e-05 2024-09-17T17:30:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346686+0000", 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0409e-05 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346782+0000", 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5859999999999994e-05 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346794+0000", 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1396999999999999e-05 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351196+0000", 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044024199999999998 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351238+0000", 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1715000000000003e-05 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 91/75 ECSubWrite(tid=202, reqid=osd.12.0:202, at_version=91'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 9.9261318599999999, 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003643519, 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9099999999999999e-07 2024-09-17T17:30:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351171+0000", 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6375999999999999e-05 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352114+0000", 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00094281599999999997 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352122+0000", 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5609999999999999e-06 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353080+0000", 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095752200000000004 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353090+0000", 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.078e-05 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354718+0000", 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001627229 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354778+0000", 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9843999999999998e-05 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 92/32 ECSubWrite(tid=282, reqid=osd.11.0:282, at_version=92'148, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.352273+0000", 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.9249927959999997, 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0031782680000000002, 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352273+0000", 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352273+0000", 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352274+0000", 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2600000000000002e-07 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352304+0000", 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0154e-05 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352477+0000", 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017361300000000001 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352487+0000", 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6949999999999995e-06 2024-09-17T17:30:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.353396+0000", 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00090919799999999995 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.353406+0000", 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0098999999999999e-05 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.355413+0000", 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020068479999999999 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.355451+0000", 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8034999999999998e-05 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 92/32 ECSubWrite(tid=283, reqid=osd.11.0:283, at_version=92'150, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:30:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 8.9249590100000002, 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039958600000000004, 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2599999999999998e-07 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352343+0000", 2024-09-17T17:30:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5757999999999999e-05 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352507+0000", 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016381600000000001 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352515+0000", 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3350000000000007e-06 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.354318+0000", 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018024930000000001 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.354329+0000", 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0991e-05 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.356257+0000", 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001928683 2024-09-17T17:30:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.356303+0000", 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5358000000000002e-05 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:30:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 3.8992852550000001, 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0042165780000000003, 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5200000000000001e-07 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378428+0000", 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044723199999999999 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378473+0000", 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5019999999999999e-05 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378492+0000", 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8980000000000001e-05 2024-09-17T17:30:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378563+0000", 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1007999999999995e-05 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378574+0000", 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0735e-05 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.382145+0000", 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003571482 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.382197+0000", 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1969000000000003e-05 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 100/76 ECSubWrite(tid=237, reqid=client.4566.0:969, at_version=100'75, trim_to=0'0, roll_forward_to=100'71))", 2024-09-17T17:30:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:51.772592+0000", 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.50467418399999997, 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046045579999999999, 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.772592+0000", 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.772592+0000", 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.772606+0000", 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3933e-05 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774378+0000", 2024-09-17T17:30:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001772266 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774410+0000", 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2135999999999999e-05 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774420+0000", 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6039999999999999e-06 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774473+0000", 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3627e-05 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774483+0000", 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.011e-05 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.777165+0000", 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002681606 2024-09-17T17:30:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.777196+0000", 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1275999999999997e-05 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.378 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:52.380 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:30:52.378+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:30:52.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":100,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:30:51.530222+0000","last_up_change":"2024-09-17T17:29:28.558396+0000","last_in_change":"2024-09-17T17:30:22.332815+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T17:29:30.150043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-17T17:29:42.633017+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":21,"pg_placement_num_target":26,"pg_num_target":26,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"67","auid":0,"snap_mode":"selfmanaged","snap_seq":34,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":0.68646240234375,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6809","nonce":1273991967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6811","nonce":1273991967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6815","nonce":1273991967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1273991967},{"type":"v1","addr":"172.21.15.7:6813","nonce":1273991967}]},"public_addr":"172.21.15.7:6809/1273991967","cluster_addr":"172.21.15.7:6811/1273991967","heartbeat_back_addr":"172.21.15.7:6815/1273991967","heartbeat_front_addr":"172.21.15.7:6813/1273991967","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":0.74237060546875,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":0.8008880615234375,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:29:25.950873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:27.557824+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:50.042615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:44.107535+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:43.930932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:46.730260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:43.862104+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:42.595312+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:40.607805+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:40.265380+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:33.448841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:29.957316+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:29.888647+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.511757+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.711104+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:32.994404+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.5","osds":[10,0,5,2,7,14,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.5","osds":[2147483647,0,5,2,7,2147483647,3]},{"pgid":"3.14","osds":[3,2147483647,2147483647,10,7,2,13]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":19,"length":1},{"begin":21,"length":2},{"begin":25,"length":7}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:30:52.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:30:52.480 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:52.479+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:52.490 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:52.491 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:52.499 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:52.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T17:30:52.593 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:52.591+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:52.609 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:52.610 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:52.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:52.682 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:52.680+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:52.703 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:52.704 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:30:52.717 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-17T17:30:52.541088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129499699,"num_objects":111,"num_object_clones":58,"num_object_copies":757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":15,"num_read":296,"num_read_kb":62485,"num_write":619,"num_write_kb":87458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":81362944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1832,"ondisk_log_size":1832,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":211,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":772760,"kb_used_data":257084,"kb_used_omap":24,"kb_used_meta":515303,"kb_avail":1509176680,"statfs":{"total":1546188226560,"available":1545396920320,"internally_reserved":0,"allocated":263254016,"data_stored":364342591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":527670351},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18481152,"num_objects":19,"num_object_clones":11,"num_object_copies":133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":22,"num_read_kb":13469,"num_write":22,"num_write_kb":8096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":52707328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.197442"},"pg_stats":[{"pgid":"3.19","version":"65'44","reported_seq":393,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T17:30:49.349579+0000","last_change":"2024-09-17T17:30:49.351146+0000","last_active":"2024-09-17T17:30:49.349579+0000","last_peered":"2024-09-17T17:30:49.349579+0000","last_clean":"2024-09-17T17:30:49.349579+0000","last_became_active":"2024-09-17T17:30:26.345748+0000","last_became_peered":"2024-09-17T17:30:26.345748+0000","last_unstale":"2024-09-17T17:30:49.349579+0000","last_undegraded":"2024-09-17T17:30:49.349579+0000","last_fullsized":"2024-09-17T17:30:49.349579+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":5,"last_scrub":"65'44","last_scrub_stamp":"2024-09-17T17:30:27.911326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.911326+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.379612+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0020145300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.18","version":"76'29","reported_seq":399,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345277+0000","last_change":"2024-09-17T17:30:47.344007+0000","last_active":"2024-09-17T17:30:48.345277+0000","last_peered":"2024-09-17T17:30:48.345277+0000","last_clean":"2024-09-17T17:30:48.345277+0000","last_became_active":"2024-09-17T17:30:27.354929+0000","last_became_peered":"2024-09-17T17:30:27.354929+0000","last_unstale":"2024-09-17T17:30:48.345277+0000","last_undegraded":"2024-09-17T17:30:48.345277+0000","last_fullsized":"2024-09-17T17:30:48.345277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":5,"last_scrub":"76'29","last_scrub_stamp":"2024-09-17T17:30:28.377344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.377344+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:40.170345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013197230000000001,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":625,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.17","version":"40'6","reported_seq":333,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380416+0000","last_change":"2024-09-17T17:30:47.352062+0000","last_active":"2024-09-17T17:30:48.380416+0000","last_peered":"2024-09-17T17:30:48.380416+0000","last_clean":"2024-09-17T17:30:48.380416+0000","last_became_active":"2024-09-17T17:30:19.460414+0000","last_became_peered":"2024-09-17T17:30:19.460414+0000","last_unstale":"2024-09-17T17:30:48.380416+0000","last_undegraded":"2024-09-17T17:30:48.380416+0000","last_fullsized":"2024-09-17T17:30:48.380416+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:25.845481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.845481+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:36.124132+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0038344030000000001,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.16","version":"97'167","reported_seq":573,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.387831+0000","last_change":"2024-09-17T17:30:47.377754+0000","last_active":"2024-09-17T17:30:48.387831+0000","last_peered":"2024-09-17T17:30:48.387831+0000","last_clean":"2024-09-17T17:30:48.387831+0000","last_became_active":"2024-09-17T17:30:47.363797+0000","last_became_peered":"2024-09-17T17:30:47.363797+0000","last_unstale":"2024-09-17T17:30:48.387831+0000","last_undegraded":"2024-09-17T17:30:48.387831+0000","last_fullsized":"2024-09-17T17:30:48.387831+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'146","last_scrub_stamp":"2024-09-17T17:30:36.055244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.055244+0000","objects_scrubbed":7,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:45.325510+0000","scrub_duration":133,"objects_trimmed":1,"snaptrim_duration":0.012742017,"stat_sum":{"num_bytes":13795328,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.15","version":"95'30","reported_seq":439,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367683+0000","last_change":"2024-09-17T17:30:47.358093+0000","last_active":"2024-09-17T17:30:48.367683+0000","last_peered":"2024-09-17T17:30:48.367683+0000","last_clean":"2024-09-17T17:30:48.367683+0000","last_became_active":"2024-09-17T17:30:46.350418+0000","last_became_peered":"2024-09-17T17:30:46.350418+0000","last_unstale":"2024-09-17T17:30:48.367683+0000","last_undegraded":"2024-09-17T17:30:48.367683+0000","last_fullsized":"2024-09-17T17:30:48.367683+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'20","last_scrub_stamp":"2024-09-17T17:30:27.352233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.352233+0000","objects_scrubbed":3,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:39.769790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0058574020000000003,"stat_sum":{"num_bytes":1736704,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":10,"num_read_kb":1250,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1736704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,2,7,11,8],"acting":[15,13,5,2,7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.14","version":"62'42","reported_seq":401,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T17:30:46.356059+0000","last_change":"2024-09-17T17:30:46.357309+0000","last_active":"2024-09-17T17:30:46.356059+0000","last_peered":"2024-09-17T17:30:46.356059+0000","last_clean":"2024-09-17T17:30:46.356059+0000","last_became_active":"2024-09-17T17:30:46.350731+0000","last_became_peered":"2024-09-17T17:30:46.350731+0000","last_unstale":"2024-09-17T17:30:46.356059+0000","last_undegraded":"2024-09-17T17:30:46.356059+0000","last_fullsized":"2024-09-17T17:30:46.356059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"62'42","last_scrub_stamp":"2024-09-17T17:30:26.686769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.686769+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.408891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050883450000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,15,9,2,6,10,13],"acting":[1,15,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367494+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:48.367494+0000","last_peered":"2024-09-17T17:30:48.367494+0000","last_clean":"2024-09-17T17:30:48.367494+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:48.367494+0000","last_undegraded":"2024-09-17T17:30:48.367494+0000","last_fullsized":"2024-09-17T17:30:48.367494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.115044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"97'139","reported_seq":518,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.376631+0000","last_change":"2024-09-17T17:30:47.377812+0000","last_active":"2024-09-17T17:30:48.376631+0000","last_peered":"2024-09-17T17:30:48.376631+0000","last_clean":"2024-09-17T17:30:48.376631+0000","last_became_active":"2024-09-17T17:30:47.362331+0000","last_became_peered":"2024-09-17T17:30:47.362331+0000","last_unstale":"2024-09-17T17:30:48.376631+0000","last_undegraded":"2024-09-17T17:30:48.376631+0000","last_fullsized":"2024-09-17T17:30:48.376631+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'127","last_scrub_stamp":"2024-09-17T17:30:27.836715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.836715+0000","objects_scrubbed":5,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:37.790365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.013564972,"stat_sum":{"num_bytes":3817472,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":6,"num_read_kb":669,"num_write":4,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367338+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:48.367338+0000","last_peered":"2024-09-17T17:30:48.367338+0000","last_clean":"2024-09-17T17:30:48.367338+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:48.367338+0000","last_undegraded":"2024-09-17T17:30:48.367338+0000","last_fullsized":"2024-09-17T17:30:48.367338+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:31.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":326,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380699+0000","last_change":"2024-09-17T17:30:47.352513+0000","last_active":"2024-09-17T17:30:48.380699+0000","last_peered":"2024-09-17T17:30:48.380699+0000","last_clean":"2024-09-17T17:30:48.380699+0000","last_became_active":"2024-09-17T17:30:19.457663+0000","last_became_peered":"2024-09-17T17:30:19.457663+0000","last_unstale":"2024-09-17T17:30:48.380699+0000","last_undegraded":"2024-09-17T17:30:48.380699+0000","last_fullsized":"2024-09-17T17:30:48.380699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:24.345150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.345150+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:45.462162+0000","scrub_duration":541,"objects_trimmed":0,"snaptrim_duration":0.0035564590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T17:30:49.347690+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:49.347690+0000","last_peered":"2024-09-17T17:30:49.347690+0000","last_clean":"2024-09-17T17:30:49.347690+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:49.347690+0000","last_undegraded":"2024-09-17T17:30:49.347690+0000","last_fullsized":"2024-09-17T17:30:49.347690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'70","reported_seq":479,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T17:30:51.551508+0000","last_change":"2024-09-17T17:30:51.552576+0000","last_active":"2024-09-17T17:30:51.551508+0000","last_peered":"2024-09-17T17:30:51.551508+0000","last_clean":"2024-09-17T17:30:51.551508+0000","last_became_active":"2024-09-17T17:30:46.352101+0000","last_became_peered":"2024-09-17T17:30:46.352101+0000","last_unstale":"2024-09-17T17:30:51.551508+0000","last_undegraded":"2024-09-17T17:30:51.551508+0000","last_fullsized":"2024-09-17T17:30:51.551508+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'48","last_scrub_stamp":"2024-09-17T17:30:27.676614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.676614+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.051338+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.015274865,"stat_sum":{"num_bytes":6848512,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13172736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,15,9,2,6,10,13],"acting":[1,15,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T17:30:51.537594+0000","last_change":"2024-09-17T17:30:47.507145+0000","last_active":"2024-09-17T17:30:51.537594+0000","last_peered":"2024-09-17T17:30:51.537594+0000","last_clean":"2024-09-17T17:30:51.537594+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:30:51.537594+0000","last_undegraded":"2024-09-17T17:30:51.537594+0000","last_fullsized":"2024-09-17T17:30:51.537594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.507074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.507074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.099148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'27","reported_seq":416,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.367929+0000","last_change":"2024-09-17T17:30:47.352462+0000","last_active":"2024-09-17T17:30:48.367929+0000","last_peered":"2024-09-17T17:30:48.367929+0000","last_clean":"2024-09-17T17:30:48.367929+0000","last_became_active":"2024-09-17T17:30:46.350505+0000","last_became_peered":"2024-09-17T17:30:46.350505+0000","last_unstale":"2024-09-17T17:30:48.367929+0000","last_undegraded":"2024-09-17T17:30:48.367929+0000","last_fullsized":"2024-09-17T17:30:48.367929+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'27","last_scrub_stamp":"2024-09-17T17:30:28.947105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.947105+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.988637+0000","scrub_duration":590,"objects_trimmed":0,"snaptrim_duration":0.00057592799999999997,"stat_sum":{"num_bytes":2113536,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,5,2,7,11,8],"acting":[15,13,5,2,7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.3","version":"97'83","reported_seq":463,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.370063+0000","last_change":"2024-09-17T17:30:47.353604+0000","last_active":"2024-09-17T17:30:48.370063+0000","last_peered":"2024-09-17T17:30:48.370063+0000","last_clean":"2024-09-17T17:30:48.370063+0000","last_became_active":"2024-09-17T17:30:26.345813+0000","last_became_peered":"2024-09-17T17:30:26.345813+0000","last_unstale":"2024-09-17T17:30:48.370063+0000","last_undegraded":"2024-09-17T17:30:48.370063+0000","last_fullsized":"2024-09-17T17:30:48.370063+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'58","last_scrub_stamp":"2024-09-17T17:30:28.942498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.942498+0000","objects_scrubbed":5,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:42.130059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001067045,"stat_sum":{"num_bytes":7880704,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":2592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.2","version":"28'2","reported_seq":159,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364427+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:48.364427+0000","last_peered":"2024-09-17T17:30:48.364427+0000","last_clean":"2024-09-17T17:30:48.364427+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:48.364427+0000","last_undegraded":"2024-09-17T17:30:48.364427+0000","last_fullsized":"2024-09-17T17:30:48.364427+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"98'92","reported_seq":465,"reported_epoch":98,"state":"active+clean+snaptrim","last_fresh":"2024-09-17T17:30:49.349658+0000","last_change":"2024-09-17T17:30:49.349658+0000","last_active":"2024-09-17T17:30:49.349658+0000","last_peered":"2024-09-17T17:30:49.349658+0000","last_clean":"2024-09-17T17:30:49.349658+0000","last_became_active":"2024-09-17T17:30:27.343750+0000","last_became_peered":"2024-09-17T17:30:27.343750+0000","last_unstale":"2024-09-17T17:30:49.349658+0000","last_undegraded":"2024-09-17T17:30:49.349658+0000","last_fullsized":"2024-09-17T17:30:49.349658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'62","last_scrub_stamp":"2024-09-17T17:30:28.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.961363+0000","objects_scrubbed":3,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:34.929359+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0096146809999999999,"stat_sum":{"num_bytes":4014080,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":10,"num_read_kb":6919,"num_write":5,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.371808+0000","last_change":"2024-09-17T17:29:33.615620+0000","last_active":"2024-09-17T17:30:48.371808+0000","last_peered":"2024-09-17T17:30:48.371808+0000","last_clean":"2024-09-17T17:30:48.371808+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:48.371808+0000","last_undegraded":"2024-09-17T17:30:48.371808+0000","last_fullsized":"2024-09-17T17:30:48.371808+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"96'47","reported_seq":413,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380032+0000","last_change":"2024-09-17T17:30:47.363646+0000","last_active":"2024-09-17T17:30:48.380032+0000","last_peered":"2024-09-17T17:30:48.380032+0000","last_clean":"2024-09-17T17:30:48.380032+0000","last_became_active":"2024-09-17T17:30:26.354825+0000","last_became_peered":"2024-09-17T17:30:26.354825+0000","last_unstale":"2024-09-17T17:30:48.380032+0000","last_undegraded":"2024-09-17T17:30:48.380032+0000","last_fullsized":"2024-09-17T17:30:48.380032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:28.852921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.852921+0000","objects_scrubbed":2,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.724409+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.018472654000000002,"stat_sum":{"num_bytes":3211264,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":8117,"num_write":6,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364635+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:30:48.364635+0000","last_peered":"2024-09-17T17:30:48.364635+0000","last_clean":"2024-09-17T17:30:48.364635+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:48.364635+0000","last_undegraded":"2024-09-17T17:30:48.364635+0000","last_fullsized":"2024-09-17T17:30:48.364635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"63'42","reported_seq":378,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.372087+0000","last_change":"2024-09-17T17:30:47.352662+0000","last_active":"2024-09-17T17:30:48.372087+0000","last_peered":"2024-09-17T17:30:48.372087+0000","last_clean":"2024-09-17T17:30:48.372087+0000","last_became_active":"2024-09-17T17:30:46.353966+0000","last_became_peered":"2024-09-17T17:30:46.353966+0000","last_unstale":"2024-09-17T17:30:48.372087+0000","last_undegraded":"2024-09-17T17:30:48.372087+0000","last_fullsized":"2024-09-17T17:30:48.372087+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'42","last_scrub_stamp":"2024-09-17T17:30:25.590903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.590903+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:36.105662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029960960000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0,3,4,15,10,11],"acting":[14,0,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.3","version":"26'1","reported_seq":166,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.371993+0000","last_change":"2024-09-17T17:30:38.649965+0000","last_active":"2024-09-17T17:30:48.371993+0000","last_peered":"2024-09-17T17:30:48.371993+0000","last_clean":"2024-09-17T17:30:48.371993+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:48.371993+0000","last_undegraded":"2024-09-17T17:30:48.371993+0000","last_fullsized":"2024-09-17T17:30:48.371993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:30:38.649873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.649873+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.468836+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345508+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:30:48.345508+0000","last_peered":"2024-09-17T17:30:48.345508+0000","last_clean":"2024-09-17T17:30:48.345508+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:48.345508+0000","last_undegraded":"2024-09-17T17:30:48.345508+0000","last_fullsized":"2024-09-17T17:30:48.345508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"75'29","reported_seq":396,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345464+0000","last_change":"2024-09-17T17:30:47.344306+0000","last_active":"2024-09-17T17:30:48.345464+0000","last_peered":"2024-09-17T17:30:48.345464+0000","last_clean":"2024-09-17T17:30:48.345464+0000","last_became_active":"2024-09-17T17:30:26.348354+0000","last_became_peered":"2024-09-17T17:30:26.348354+0000","last_unstale":"2024-09-17T17:30:48.345464+0000","last_undegraded":"2024-09-17T17:30:48.345464+0000","last_fullsized":"2024-09-17T17:30:48.345464+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'29","last_scrub_stamp":"2024-09-17T17:30:27.407808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.407808+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:53.792072+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.001223395,"stat_sum":{"num_bytes":344064,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.9","version":"98'69","reported_seq":465,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T17:30:49.365392+0000","last_change":"2024-09-17T17:30:49.365957+0000","last_active":"2024-09-17T17:30:49.365392+0000","last_peered":"2024-09-17T17:30:49.365392+0000","last_clean":"2024-09-17T17:30:49.365392+0000","last_became_active":"2024-09-17T17:30:28.394697+0000","last_became_peered":"2024-09-17T17:30:28.394697+0000","last_unstale":"2024-09-17T17:30:49.365392+0000","last_undegraded":"2024-09-17T17:30:49.365392+0000","last_fullsized":"2024-09-17T17:30:49.365392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'61","last_scrub_stamp":"2024-09-17T17:30:36.024221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.024221+0000","objects_scrubbed":4,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:56.003220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.016735380000000001,"stat_sum":{"num_bytes":7012352,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":12170,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.a","version":"92'136","reported_seq":451,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364231+0000","last_change":"2024-09-17T17:30:47.348842+0000","last_active":"2024-09-17T17:30:48.364231+0000","last_peered":"2024-09-17T17:30:48.364231+0000","last_clean":"2024-09-17T17:30:48.364231+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:48.364231+0000","last_undegraded":"2024-09-17T17:30:48.364231+0000","last_fullsized":"2024-09-17T17:30:48.364231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:49.643669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001344788,"stat_sum":{"num_bytes":14680064,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":52,"num_read_kb":15272,"num_write":108,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.b","version":"96'22","reported_seq":361,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.380199+0000","last_change":"2024-09-17T17:30:47.372788+0000","last_active":"2024-09-17T17:30:48.380199+0000","last_peered":"2024-09-17T17:30:48.380199+0000","last_clean":"2024-09-17T17:30:48.380199+0000","last_became_active":"2024-09-17T17:30:47.358299+0000","last_became_peered":"2024-09-17T17:30:47.358299+0000","last_unstale":"2024-09-17T17:30:48.380199+0000","last_undegraded":"2024-09-17T17:30:48.380199+0000","last_fullsized":"2024-09-17T17:30:48.380199+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:08.923689+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013131362000000001,"stat_sum":{"num_bytes":2310144,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":6719,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4538368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,10,6,8,7,14],"acting":[13,3,10,6,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.c","version":"97'156","reported_seq":463,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.379310+0000","last_change":"2024-09-17T17:30:47.347763+0000","last_active":"2024-09-17T17:30:48.379310+0000","last_peered":"2024-09-17T17:30:48.379310+0000","last_clean":"2024-09-17T17:30:48.379310+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:48.379310+0000","last_undegraded":"2024-09-17T17:30:48.379310+0000","last_fullsized":"2024-09-17T17:30:48.379310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002112245,"stat_sum":{"num_bytes":17776640,"num_objects":13,"num_object_clones":8,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":45,"num_read_kb":3633,"num_write":101,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.d","version":"96'97","reported_seq":389,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T17:30:47.353500+0000","last_change":"2024-09-17T17:30:47.354659+0000","last_active":"2024-09-17T17:30:47.353500+0000","last_peered":"2024-09-17T17:30:47.353500+0000","last_clean":"2024-09-17T17:30:47.353500+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:30:47.353500+0000","last_undegraded":"2024-09-17T17:30:47.353500+0000","last_fullsized":"2024-09-17T17:30:47.353500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:08.625403+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0094019519999999999,"stat_sum":{"num_bytes":9322496,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":15,"num_read_kb":3611,"num_write":58,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.e","version":"98'64","reported_seq":376,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T17:30:49.368482+0000","last_change":"2024-09-17T17:30:49.369290+0000","last_active":"2024-09-17T17:30:49.368482+0000","last_peered":"2024-09-17T17:30:49.368482+0000","last_clean":"2024-09-17T17:30:49.368482+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:49.368482+0000","last_undegraded":"2024-09-17T17:30:49.368482+0000","last_fullsized":"2024-09-17T17:30:49.368482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:08.779401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.02157796,"stat_sum":{"num_bytes":6520832,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":18,"num_read_kb":3129,"num_write":52,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.f","version":"96'94","reported_seq":494,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364480+0000","last_change":"2024-09-17T17:30:47.372219+0000","last_active":"2024-09-17T17:30:48.364480+0000","last_peered":"2024-09-17T17:30:48.364480+0000","last_clean":"2024-09-17T17:30:48.364480+0000","last_became_active":"2024-09-17T17:30:47.356042+0000","last_became_peered":"2024-09-17T17:30:47.356042+0000","last_unstale":"2024-09-17T17:30:48.364480+0000","last_undegraded":"2024-09-17T17:30:48.364480+0000","last_fullsized":"2024-09-17T17:30:48.364480+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.145356+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01447214,"stat_sum":{"num_bytes":15581184,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":74,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":41320448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,4,14,2,3,0],"acting":[11,8,4,14,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.10","version":"77'53","reported_seq":159,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.345149+0000","last_change":"2024-09-17T17:30:47.356852+0000","last_active":"2024-09-17T17:30:48.345149+0000","last_peered":"2024-09-17T17:30:48.345149+0000","last_clean":"2024-09-17T17:30:48.345149+0000","last_became_active":"2024-09-17T17:30:47.354125+0000","last_became_peered":"2024-09-17T17:30:47.354125+0000","last_unstale":"2024-09-17T17:30:48.345149+0000","last_undegraded":"2024-09-17T17:30:48.345149+0000","last_fullsized":"2024-09-17T17:30:48.345149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'52","last_scrub_stamp":"2024-09-17T17:30:26.907313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.907313+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:32.936798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001721717,"stat_sum":{"num_bytes":2342912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7,14,5,9,11],"acting":[6,0,7,14,5,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.11","version":"60'25","reported_seq":106,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T17:30:49.354284+0000","last_change":"2024-09-17T17:30:49.355374+0000","last_active":"2024-09-17T17:30:49.354284+0000","last_peered":"2024-09-17T17:30:49.354284+0000","last_clean":"2024-09-17T17:30:49.354284+0000","last_became_active":"2024-09-17T17:30:36.353414+0000","last_became_peered":"2024-09-17T17:30:36.353414+0000","last_unstale":"2024-09-17T17:30:49.354284+0000","last_undegraded":"2024-09-17T17:30:49.354284+0000","last_fullsized":"2024-09-17T17:30:49.354284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:37.309789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:37.309789+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:53.545628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,10,5,14,2],"acting":[4,11,15,10,5,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.12","version":"93'84","reported_seq":471,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.371976+0000","last_change":"2024-09-17T17:30:47.350338+0000","last_active":"2024-09-17T17:30:48.371976+0000","last_peered":"2024-09-17T17:30:48.371976+0000","last_clean":"2024-09-17T17:30:48.371976+0000","last_became_active":"2024-09-17T17:30:39.350802+0000","last_became_peered":"2024-09-17T17:30:39.350802+0000","last_unstale":"2024-09-17T17:30:48.371976+0000","last_undegraded":"2024-09-17T17:30:48.371976+0000","last_fullsized":"2024-09-17T17:30:48.371976+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'48","last_scrub_stamp":"2024-09-17T17:30:24.093396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.093396+0000","objects_scrubbed":3,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.256082+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0.0010622699999999999,"stat_sum":{"num_bytes":3047424,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":9,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6160384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,7,5,4,11,3,15],"acting":[14,7,5,4,11,3,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.13","version":"83'64","reported_seq":465,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T17:30:48.364524+0000","last_change":"2024-09-17T17:30:47.348278+0000","last_active":"2024-09-17T17:30:48.364524+0000","last_peered":"2024-09-17T17:30:48.364524+0000","last_clean":"2024-09-17T17:30:48.364524+0000","last_became_active":"2024-09-17T17:30:46.350583+0000","last_became_peered":"2024-09-17T17:30:46.350583+0000","last_unstale":"2024-09-17T17:30:48.364524+0000","last_undegraded":"2024-09-17T17:30:48.364524+0000","last_fullsized":"2024-09-17T17:30:48.364524+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'52","last_scrub_stamp":"2024-09-17T17:30:27.908152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.908152+0000","objects_scrubbed":2,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:47.701212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0022866409999999998,"stat_sum":{"num_bytes":4063232,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7634944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,6,5,4,13],"acting":[11,7,2,6,5,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"19","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":128909312,"num_objects":107,"num_object_clones":58,"num_object_copies":749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":15,"num_read":210,"num_read_kb":62412,"num_write":472,"num_write_kb":84784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":81362944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":240730112,"data_stored":354160640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1717,"ondisk_log_size":1717,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247827,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46836,"kb_used_data":11488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325004,"statfs":{"total":96636764160,"available":96588804096,"internally_reserved":0,"allocated":11763712,"data_stored":20098560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":19612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":20082688,"data_stored":29804015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247828,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46760,"kb_used_data":15572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325080,"statfs":{"total":96636764160,"available":96588881920,"internally_reserved":0,"allocated":15945728,"data_stored":15816381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313236,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51424,"kb_used_data":16068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94320416,"statfs":{"total":96636764160,"available":96584105984,"internally_reserved":0,"allocated":16453632,"data_stored":23678834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44860,"kb_used_data":13672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326980,"statfs":{"total":96636764160,"available":96590827520,"internally_reserved":0,"allocated":14000128,"data_stored":21587439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48264,"kb_used_data":17072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323576,"statfs":{"total":96636764160,"available":96587341824,"internally_reserved":0,"allocated":17481728,"data_stored":23653716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280532,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":19164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":19623936,"data_stored":23320591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46680,"kb_used_data":15488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325160,"statfs":{"total":96636764160,"available":96588963840,"internally_reserved":0,"allocated":15859712,"data_stored":22726127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49544,"kb_used_data":14188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322296,"statfs":{"total":96636764160,"available":96586031104,"internally_reserved":0,"allocated":14528512,"data_stored":18896367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280532,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48300,"kb_used_data":12952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323540,"statfs":{"total":96636764160,"available":96587304960,"internally_reserved":0,"allocated":13262848,"data_stored":21966355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280531,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42888,"kb_used_data":11700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328952,"statfs":{"total":96636764160,"available":96592846848,"internally_reserved":0,"allocated":11980800,"data_stored":17966575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,7,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46152,"kb_used_data":14964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325688,"statfs":{"total":96636764160,"available":96589504512,"internally_reserved":0,"allocated":15323136,"data_stored":20207087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47212,"kb_used_data":20180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94324628,"statfs":{"total":96636764160,"available":96588419072,"internally_reserved":0,"allocated":20664320,"data_stored":31000047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53292,"kb_used_data":22104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318548,"statfs":{"total":96636764160,"available":96582193152,"internally_reserved":0,"allocated":22634496,"data_stored":26220034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247828,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50632,"kb_used_data":15284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321208,"statfs":{"total":96636764160,"available":96584916992,"internally_reserved":0,"allocated":15650816,"data_stored":25792000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247827,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48764,"kb_used_data":17576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323076,"statfs":{"total":96636764160,"available":96586829824,"internally_reserved":0,"allocated":17997824,"data_stored":21608463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17825792,"data_stored":22319104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14991360,"data_stored":23920640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19464192,"data_stored":31150080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21430272,"data_stored":26370048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14221312,"data_stored":24956928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13185024,"data_stored":18141184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16199680,"data_stored":20606976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16064512,"data_stored":21340160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14585856,"data_stored":15056896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":11829248,"data_stored":21131264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10633216,"data_stored":17764352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":18739200,"data_stored":29048832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":10334208,"data_stored":19263488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":14516224,"data_stored":21970944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":12734464,"data_stored":21114880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":13975552,"data_stored":20004864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:30:52.720 INFO:tasks.ceph.ceph_manager.ceph:pg 3.0 in trimming, state: active+clean+snaptrim 2024-09-17T17:30:52.720 INFO:tasks.ceph.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-09-17T17:30:52.728 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:182 3.7s0 3:ea2a6e9a:::smithi10238019-22:head [copy-from ver 8] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 62.047414777999997, 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.030026536, 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 182 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677857+0000", 2024-09-17T17:30:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2839999999999999e-06 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677863+0000", 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8850000000000004e-06 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677875+0000", 2024-09-17T17:30:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2490999999999999e-05 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677882+0000", 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.652e-06 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677943+0000", 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1327000000000004e-05 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678017+0000", 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3440000000000002e-05 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703307+0000", 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.025289977000000002 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704581+0000", 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012741790000000001 2024-09-17T17:30:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707797+0000", 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003216642 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707882+0000", 2024-09-17T17:30:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4659000000000004e-05 2024-09-17T17:30:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:183 3.7s0 3:ea2a6e9a:::smithi10238019-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 62.047376065999998, 2024-09-17T17:30:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026638608000000001, 2024-09-17T17:30:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 183 2024-09-17T17:30:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:30:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2099999999999998e-07 2024-09-17T17:30:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677896+0000", 2024-09-17T17:30:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.804e-06 2024-09-17T17:30:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677902+0000", 2024-09-17T17:30:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5149999999999997e-06 2024-09-17T17:30:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677905+0000", 2024-09-17T17:30:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5209999999999998e-06 2024-09-17T17:30:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678152+0000", 2024-09-17T17:30:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000246536 2024-09-17T17:30:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678180+0000", 2024-09-17T17:30:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8605000000000001e-05 2024-09-17T17:30:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704308+0000", 2024-09-17T17:30:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026127128999999999 2024-09-17T17:30:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704406+0000", 2024-09-17T17:30:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8079999999999996e-05 2024-09-17T17:30:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704533+0000", 2024-09-17T17:30:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012709700000000001 2024-09-17T17:30:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 43/32 ECSubWriteReply(tid=93, last_complete=43'8, committed=1, applied=1))", 2024-09-17T17:30:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:30:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.018141829999998, 2024-09-17T17:30:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.427260368, 2024-09-17T17:30:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:30:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:30:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707129+0000", 2024-09-17T17:30:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9599999999999999e-07 2024-09-17T17:30:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707133+0000", 2024-09-17T17:30:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0629999999999999e-06 2024-09-17T17:30:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134286+0000", 2024-09-17T17:30:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.427152792 2024-09-17T17:30:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134295+0000", 2024-09-17T17:30:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9500000000000007e-06 2024-09-17T17:30:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134356+0000", 2024-09-17T17:30:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1439000000000006e-05 2024-09-17T17:30:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134389+0000", 2024-09-17T17:30:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2428000000000002e-05 2024-09-17T17:30:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:489 3.1s0 3:87ca9cdd:::smithi10238019-41:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:30:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.355756132000003, 2024-09-17T17:30:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01402203, 2024-09-17T17:30:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 489 2024-09-17T17:30:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:30:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:30:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369517+0000", 2024-09-17T17:30:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8480000000000001e-06 2024-09-17T17:30:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369527+0000", 2024-09-17T17:30:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0037000000000001e-05 2024-09-17T17:30:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369548+0000", 2024-09-17T17:30:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0653e-05 2024-09-17T17:30:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369556+0000", 2024-09-17T17:30:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5909999999999996e-06 2024-09-17T17:30:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369630+0000", 2024-09-17T17:30:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3579999999999997e-05 2024-09-17T17:30:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "op must wait for map", 2024-09-17T17:30:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369636+0000", 2024-09-17T17:30:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6900000000000003e-06 2024-09-17T17:30:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373208+0000", 2024-09-17T17:30:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0035712719999999999 2024-09-17T17:30:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374240+0000", 2024-09-17T17:30:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010317569999999999 2024-09-17T17:30:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375226+0000", 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00098623599999999993 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.380028+0000", 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0048024870000000002 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.383466+0000", 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0034380299999999999 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.383536+0000", 2024-09-17T17:30:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9849000000000002e-05 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:490 3.1s0 3:87ca9cdd:::smithi10238019-41:head [rollback 9] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.355692328000003, 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016574469000000001, 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 490 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5999999999999999e-07 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369581+0000", 2024-09-17T17:30:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5000000000000002e-06 2024-09-17T17:30:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369586+0000", 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7570000000000004e-06 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369589+0000", 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7340000000000002e-06 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369733+0000", 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014407800000000001 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369741+0000", 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8310000000000001e-06 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.376615+0000", 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0068736020000000004 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.376737+0000", 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012183199999999999 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377593+0000", 2024-09-17T17:30:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00085658399999999997 2024-09-17T17:30:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.381579+0000", 2024-09-17T17:30:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039858879999999999 2024-09-17T17:30:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386071+0000", 2024-09-17T17:30:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044917280000000004 2024-09-17T17:30:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386152+0000", 2024-09-17T17:30:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1475000000000002e-05 2024-09-17T17:30:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:491 3.1s0 3:87ca9cdd:::smithi10238019-41:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.355653175, 2024-09-17T17:30:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016871317, 2024-09-17T17:30:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 491 2024-09-17T17:30:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:30:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8799999999999998e-07 2024-09-17T17:30:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369623+0000", 2024-09-17T17:30:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0730000000000004e-06 2024-09-17T17:30:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369629+0000", 2024-09-17T17:30:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0310000000000004e-06 2024-09-17T17:30:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369633+0000", 2024-09-17T17:30:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9899999999999999e-06 2024-09-17T17:30:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369901+0000", 2024-09-17T17:30:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00026797300000000002 2024-09-17T17:30:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369915+0000", 2024-09-17T17:30:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4143e-05 2024-09-17T17:30:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.378961+0000", 2024-09-17T17:30:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0090466739999999993 2024-09-17T17:30:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379126+0000", 2024-09-17T17:30:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016480800000000001 2024-09-17T17:30:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379682+0000", 2024-09-17T17:30:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00055593299999999999 2024-09-17T17:30:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.381583+0000", 2024-09-17T17:30:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001901123 2024-09-17T17:30:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386406+0000", 2024-09-17T17:30:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0048232199999999996 2024-09-17T17:30:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386488+0000", 2024-09-17T17:30:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1960999999999997e-05 2024-09-17T17:30:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:492 3.bs0 3:d4565dea:::smithi10238019-24:head [delete] snapc c=[9] ondisk+write+known_if_redirected+full_force+supports_pool_eio e57)", 2024-09-17T17:30:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:30:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.30155259, 2024-09-17T17:30:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.23375553299999999, 2024-09-17T17:30:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 492 2024-09-17T17:30:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423720+0000", 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4530000000000001e-06 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423731+0000", 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1127e-05 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423762+0000", 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1028999999999997e-05 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423777+0000", 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4357e-05 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423949+0000", 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017206400000000001 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.424109+0000", 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000160703 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.424863+0000", 2024-09-17T17:30:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00075321999999999997 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.425916+0000", 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001053888 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.657390+0000", 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.231473764 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.657473+0000", 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2928000000000003e-05 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:582 3.6s0 3:672dafef:::smithi10238019-1:head [copy-from ver 5] snapc 11=[11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:30:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.363315868999997, 2024-09-17T17:30:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019590435999999999, 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 582 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361956+0000", 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4789999999999999e-06 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361969+0000", 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2979999999999999e-05 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361983+0000", 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3917000000000001e-05 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361992+0000", 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8850000000000005e-06 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364412+0000", 2024-09-17T17:30:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002420082 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364470+0000", 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8588999999999998e-05 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.379219+0000", 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014748225 2024-09-17T17:30:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380383+0000", 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011644050000000001 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.381488+0000", 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001105225 2024-09-17T17:30:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.381545+0000", 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6648999999999999e-05 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:583 3.6s0 3:672dafef:::smithi10238019-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 36.363241969000001, 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018119641999999998, 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 583 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362029+0000", 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5799999999999999e-07 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362032+0000", 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2729999999999998e-06 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362040+0000", 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5650000000000004e-06 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362044+0000", 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5549999999999996e-06 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364703+0000", 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026589909999999999 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364754+0000", 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0785999999999998e-05 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380012+0000", 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015258315 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380076+0000", 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.368e-05 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380148+0000", 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1919e-05 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 68/67 ECSubWriteReply(tid=199, last_complete=68'121, committed=1, applied=1))", 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:30:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 33.221279809999999, 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.188788604, 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503991+0000", 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7700000000000003e-07 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503996+0000", 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4039999999999997e-06 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692049+0000", 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.188053784 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692058+0000", 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1480000000000006e-06 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692105+0000", 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6878000000000001e-05 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692779+0000", 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067451299999999996 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/67 ECSubWriteReply(tid=202, last_complete=68'119, committed=1, applied=1))", 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 33.025398787999997, 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.16860020100000001, 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699872+0000", 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2699999999999998e-07 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699877+0000", 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7090000000000001e-06 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868220+0000", 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.168342782 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868242+0000", 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1985999999999999e-05 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868400+0000", 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000158331 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868472+0000", 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1565999999999999e-05 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 70/67 ECSubWriteReply(tid=208, last_complete=70'125, committed=1, applied=1))", 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 31.361269129, 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.158690363, 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364002+0000", 2024-09-17T17:30:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1199999999999998e-07 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364005+0000", 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.117e-06 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522493+0000", 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15848867899999999 2024-09-17T17:30:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522508+0000", 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4419999999999999e-05 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522668+0000", 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016041699999999999 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522691+0000", 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3317999999999999e-05 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 70/67 ECSubWriteReply(tid=208, last_complete=70'125, committed=1, applied=1))", 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:30:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 31.356908142999998, 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.71001705199999998, 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368363+0000", 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7199999999999997e-07 2024-09-17T17:30:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368367+0000", 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3680000000000004e-06 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078255+0000", 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.70988762599999999 2024-09-17T17:30:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078263+0000", 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7430000000000002e-06 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078349+0000", 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6163999999999997e-05 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078379+0000", 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0278999999999999e-05 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 75/74 ECSubWriteReply(tid=239, last_complete=75'130, committed=1, applied=1))", 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 26.348906719999999, 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.20952685800000001, 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.68e-07 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376368+0000", 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585670+0000", 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.20930257699999999 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585686+0000", 2024-09-17T17:30:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5676e-05 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585835+0000", 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014933399999999999 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585890+0000", 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4969999999999997e-05 2024-09-17T17:30:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:796 3.16s0 3:6f166faa:::smithi10238019-38:head [copy-from ver 25] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.364298304999998, 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.020945305000000001, 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 796 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.01e-07 2024-09-17T17:30:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360975+0000", 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5620000000000002e-06 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360981+0000", 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4989999999999999e-06 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360988+0000", 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3369999999999996e-06 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361038+0000", 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0141e-05 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361130+0000", 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1666e-05 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.374988+0000", 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01385871 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.377231+0000", 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022428610000000001 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.381852+0000", 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004620613 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.381917+0000", 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5514999999999999e-05 2024-09-17T17:30:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:797 3.16s0 3:6f166faa:::smithi10238019-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.364271233, 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016029564999999999, 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 797 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4700000000000002e-07 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361002+0000", 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3929999999999998e-06 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361008+0000", 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6470000000000001e-06 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361014+0000", 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4650000000000002e-06 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361241+0000", 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000227035 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361277+0000", 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6492000000000002e-05 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.376753+0000", 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015475401999999999 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.376852+0000", 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9086999999999995e-05 2024-09-17T17:30:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.377029+0000", 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017659699999999999 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:841 3.16s0 3:6f166faa:::smithi10238019-38:head [copy-from ver 27] snapc 1c=[1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:30:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.339062+0000", 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.386207755000001, 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.053373410000000003, 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 841 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339062+0000", 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339062+0000", 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339065+0000", 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339075+0000", 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0540000000000001e-05 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339095+0000", 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9737000000000001e-05 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339105+0000", 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0526000000000001e-05 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339197+0000", 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1257999999999999e-05 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "op must wait for map", 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339206+0000", 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0599999999999997e-06 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.341465+0000", 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002258917 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.341601+0000", 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013603199999999999 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.363580+0000", 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.02197928 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.366338+0000", 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027583389999999998 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.392376+0000", 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026037919999999999 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.392436+0000", 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9599000000000003e-05 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:842 3.16s0 3:6f166faa:::smithi10238019-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.386143520000001, 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026888496000000001, 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 842 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3700000000000001e-07 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339130+0000", 2024-09-17T17:30:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5129999999999999e-06 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339137+0000", 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0870000000000001e-06 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339141+0000", 2024-09-17T17:30:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0230000000000003e-06 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339270+0000", 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000129326 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339274+0000", 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9700000000000001e-06 2024-09-17T17:30:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.342500+0000", 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032258740000000001 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.342566+0000", 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6150999999999997e-05 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.365784+0000", 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.023217987999999998 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.365901+0000", 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000116741 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.366015+0000", 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000114386 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:847 3.16s0 3:6d14b604:::smithi10238019-45:head [setxattr _header (57) in=7b] snapc 1c=[1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.400677+0000", 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.324592836000001, 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.12075385900000001, 2024-09-17T17:30:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 847 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400677+0000", 2024-09-17T17:30:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400677+0000", 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400678+0000", 2024-09-17T17:30:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2399999999999998e-07 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400682+0000", 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0509999999999998e-06 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400687+0000", 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7999999999999998e-06 2024-09-17T17:30:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400690+0000", 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3689999999999998e-06 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402064+0000", 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013741160000000001 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402168+0000", 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010328 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402694+0000", 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00052647200000000005 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.403384+0000", 2024-09-17T17:30:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068960500000000004 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521353+0000", 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.11796981600000001 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521431+0000", 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7825999999999997e-05 2024-09-17T17:30:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:848 3.16s0 3:6d14b604:::smithi10238019-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.400695+0000", 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.324574987999998, 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.121926795, 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 848 2024-09-17T17:30:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400695+0000", 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400695+0000", 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400696+0000", 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3099999999999999e-07 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400697+0000", 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7039999999999999e-06 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400702+0000", 2024-09-17T17:30:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5310000000000003e-06 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400705+0000", 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1750000000000001e-06 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402985+0000", 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022796460000000002 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.403051+0000", 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6756999999999998e-05 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521810+0000", 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.118758407 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521912+0000", 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000101847 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.522622+0000", 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00071039699999999998 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:52.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:52.895 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:52.893+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:52.995 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:52.993+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:53.014 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:53.014 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:53.014 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:53.015 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:53.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:30:53.096 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:53.094+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:53.131 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:53.132 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:53.132 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:53.132 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:53.132 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:53.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:30:53.197 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:53.195+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:53.248 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:53.249 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:53.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:53.249 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:53.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.250 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 75/74 ECSubWrite(tid=235, reqid=osd.13.0:235, at_version=75'127, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:53.250 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:53.250 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.885104133999999, 2024-09-17T17:30:53.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069013160000000002, 2024-09-17T17:30:53.250 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:53.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1200000000000003e-07 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361014+0000", 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4908e-05 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361042+0000", 2024-09-17T17:30:53.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8481000000000001e-05 2024-09-17T17:30:53.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361050+0000", 2024-09-17T17:30:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7109999999999995e-06 2024-09-17T17:30:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361102+0000", 2024-09-17T17:30:53.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1959999999999997e-05 2024-09-17T17:30:53.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361112+0000", 2024-09-17T17:30:53.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0345e-05 2024-09-17T17:30:53.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.367865+0000", 2024-09-17T17:30:53.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067524300000000002 2024-09-17T17:30:53.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.367900+0000", 2024-09-17T17:30:53.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4968999999999997e-05 2024-09-17T17:30:53.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 75/74 ECSubWrite(tid=236, reqid=osd.13.0:236, at_version=75'128, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:53.259 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.883125067999998, 2024-09-17T17:30:53.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077947959999999997, 2024-09-17T17:30:53.259 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.259 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:53.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:30:53.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-17T17:30:53.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362994+0000", 2024-09-17T17:30:53.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5650000000000001e-05 2024-09-17T17:30:53.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363018+0000", 2024-09-17T17:30:53.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4300000000000001e-05 2024-09-17T17:30:53.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363026+0000", 2024-09-17T17:30:53.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4540000000000001e-06 2024-09-17T17:30:53.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363078+0000", 2024-09-17T17:30:53.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2086999999999998e-05 2024-09-17T17:30:53.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363088+0000", 2024-09-17T17:30:53.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.005e-05 2024-09-17T17:30:53.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370737+0000", 2024-09-17T17:30:53.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076488509999999999 2024-09-17T17:30:53.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370772+0000", 2024-09-17T17:30:53.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5858999999999997e-05 2024-09-17T17:30:53.268 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.268 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.268 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.269 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:53.269 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:53.269 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.882266882, 2024-09-17T17:30:53.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095484720000000006, 2024-09-17T17:30:53.269 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2799999999999996e-07 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363852+0000", 2024-09-17T17:30:53.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515e-05 2024-09-17T17:30:53.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363883+0000", 2024-09-17T17:30:53.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1594999999999998e-05 2024-09-17T17:30:53.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363893+0000", 2024-09-17T17:30:53.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:53.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365558+0000", 2024-09-17T17:30:53.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001665281 2024-09-17T17:30:53.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365563+0000", 2024-09-17T17:30:53.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3410000000000002e-06 2024-09-17T17:30:53.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373353+0000", 2024-09-17T17:30:53.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077893490000000001 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373384+0000", 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1711999999999997e-05 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=106, reqid=client.4566.0:786, at_version=79'58, trim_to=0'0, roll_forward_to=78'54))", 2024-09-17T17:30:53.278 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.88518921, 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0061729920000000004, 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360915+0000", 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1769999999999999e-06 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361086+0000", 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017148 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361114+0000", 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7773999999999999e-05 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361124+0000", 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0380999999999999e-05 2024-09-17T17:30:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361197+0000", 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3140999999999996e-05 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361218+0000", 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0197e-05 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367041+0000", 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005823249 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367086+0000", 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5593000000000002e-05 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=107, reqid=client.4566.0:787, at_version=79'59, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.883219558, 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0070290120000000003, 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362884+0000", 2024-09-17T17:30:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.037e-06 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362968+0000", 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3614000000000003e-05 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363001+0000", 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3385e-05 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363013+0000", 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1761e-05 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363106+0000", 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2560999999999997e-05 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363117+0000", 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1457999999999999e-05 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369881+0000", 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006764417 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369912+0000", 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0778999999999998e-05 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 91/76 ECSubWrite(tid=201, reqid=osd.12.0:201, at_version=91'65, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.892350925000001, 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060328960000000003, 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:30:53.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4900000000000001e-07 2024-09-17T17:30:53.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353782+0000", 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9933999999999998e-05 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353905+0000", 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012266299999999999 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353914+0000", 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2050000000000008e-06 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.354626+0000", 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071210800000000003 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.354639+0000", 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2594e-05 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.359660+0000", 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050218559999999999 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.359785+0000", 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000124187 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=202, reqid=osd.12.0:202, at_version=91'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.892317666, 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0082767129999999998, 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5100000000000001e-07 2024-09-17T17:30:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353809+0000", 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4088999999999999e-05 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356309+0000", 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024992579999999999 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356319+0000", 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7629999999999995e-06 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356466+0000", 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147269 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356477+0000", 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1569000000000001e-05 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.362024+0000", 2024-09-17T17:30:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055471019999999999 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.362062+0000", 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7311999999999998e-05 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=203, reqid=client.4566.0:889, at_version=91'75, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.355806+0000", 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.890296466000001, 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012494346, 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.355806+0000", 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.355806+0000", 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.355807+0000", 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356290+0000", 2024-09-17T17:30:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048311300000000002 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357481+0000", 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011908070000000001 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357491+0000", 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7659999999999993e-06 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.358479+0000", 2024-09-17T17:30:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00098815800000000005 2024-09-17T17:30:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.358489+0000", 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0084999999999999e-05 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368256+0000", 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0097670750000000001 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368301+0000", 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4666000000000002e-05 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=204, reqid=client.4566.0:890, at_version=91'76, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.357411+0000", 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.888691264, 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01092184, 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357411+0000", 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357411+0000", 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357413+0000", 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0860000000000001e-06 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357458+0000", 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5321999999999997e-05 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.360367+0000", 2024-09-17T17:30:53.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029088289999999999 2024-09-17T17:30:53.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.360376+0000", 2024-09-17T17:30:53.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6160000000000008e-06 2024-09-17T17:30:53.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.361787+0000", 2024-09-17T17:30:53.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014109420000000001 2024-09-17T17:30:53.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.361806+0000", 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8380999999999999e-05 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368305+0000", 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064998740000000001 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368333+0000", 2024-09-17T17:30:53.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.779e-05 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=206, reqid=client.4566.0:890, at_version=0'0, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:30:53.313 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.877132016000001, 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010717031, 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:30:53.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:30:53.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368984+0000", 2024-09-17T17:30:53.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2493e-05 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373159+0000", 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041748449999999999 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373167+0000", 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2700000000000004e-06 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373249+0000", 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1790000000000001e-05 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373260+0000", 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.115e-05 2024-09-17T17:30:53.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.379654+0000", 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0063941529999999996 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.379688+0000", 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3578999999999998e-05 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.320 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.321 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 91/74 ECSubWrite(tid=307, reqid=client.4566.0:903, at_version=0'0, trim_to=0'0, roll_forward_to=91'133))", 2024-09-17T17:30:53.321 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.372644+0000", 2024-09-17T17:30:53.321 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.873458369, 2024-09-17T17:30:53.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078174990000000003, 2024-09-17T17:30:53.322 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.322 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372644+0000", 2024-09-17T17:30:53.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372644+0000", 2024-09-17T17:30:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372645+0000", 2024-09-17T17:30:53.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2399999999999997e-07 2024-09-17T17:30:53.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372657+0000", 2024-09-17T17:30:53.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.182e-05 2024-09-17T17:30:53.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372675+0000", 2024-09-17T17:30:53.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.77e-05 2024-09-17T17:30:53.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372709+0000", 2024-09-17T17:30:53.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4307999999999997e-05 2024-09-17T17:30:53.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372793+0000", 2024-09-17T17:30:53.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4443000000000004e-05 2024-09-17T17:30:53.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372937+0000", 2024-09-17T17:30:53.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014369299999999999 2024-09-17T17:30:53.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380433+0000", 2024-09-17T17:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.007495832 2024-09-17T17:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380462+0000", 2024-09-17T17:30:53.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8979000000000002e-05 2024-09-17T17:30:53.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.332 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.0s3 91/75 ECSubRead(tid=210, to_read={3:00c2d3b1:::smithi10238019-29:head=204800,258048,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:53.332 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:30:53.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.872261503000001, 2024-09-17T17:30:53.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.94432460600000001, 2024-09-17T17:30:53.333 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.333 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:30:53.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:30:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373845+0000", 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9469999999999996e-06 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.317764+0000", 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.94391877700000004 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.317783+0000", 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9037000000000001e-05 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.317920+0000", 2024-09-17T17:30:53.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000137059 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.318166+0000", 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245541 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 92/75 ECSubWrite(tid=211, reqid=osd.12.0:211, at_version=92'78, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:30:53.338 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.8925667080000004, 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067417500000000003, 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353575+0000", 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8878999999999998e-05 2024-09-17T17:30:53.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353655+0000", 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9399000000000003e-05 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353667+0000", 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2383e-05 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353745+0000", 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7644000000000004e-05 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353757+0000", 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2074e-05 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.360229+0000", 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064723640000000004 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.360278+0000", 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8773999999999998e-05 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 92/75 ECSubWrite(tid=212, reqid=osd.12.0:212, at_version=92'80, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.8921371150000006, 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0071408280000000001, 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5700000000000002e-07 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354034+0000", 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7350999999999999e-05 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354071+0000", 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6807000000000002e-05 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354079+0000", 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4270000000000008e-06 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354185+0000", 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105991 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354195+0000", 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8460000000000003e-06 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361063+0000", 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068683379999999999 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361106+0000", 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3310999999999999e-05 2024-09-17T17:30:53.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s6 92/74 ECSubWrite(tid=316, reqid=osd.13.0:316, at_version=92'34, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.355682+0000", 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.8904202409999993, 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060278060000000001, 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355682+0000", 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355682+0000", 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355686+0000", 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.461e-06 2024-09-17T17:30:53.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355713+0000", 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.707e-05 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355795+0000", 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1611000000000002e-05 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355803+0000", 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1440000000000003e-06 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355897+0000", 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4637999999999998e-05 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355910+0000", 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2306e-05 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361664+0000", 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057543170000000001 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361710+0000", 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6258999999999999e-05 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.352 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.3s5 92/74 ECSubWrite(tid=128, reqid=osd.10.0:128, at_version=92'78, trim_to=0'0, roll_forward_to=85'76))", 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.890137008, 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0098776119999999992, 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.34e-07 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356001+0000", 2024-09-17T17:30:53.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5017e-05 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356088+0000", 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6717000000000003e-05 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356096+0000", 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.208e-06 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356167+0000", 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1194999999999999e-05 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356177+0000", 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7089999999999997e-06 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.365807+0000", 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0096299200000000001 2024-09-17T17:30:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.365843+0000", 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6511999999999999e-05 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 92/76 ECSubWrite(tid=213, reqid=osd.12.0:213, at_version=92'67, trim_to=0'0, roll_forward_to=91'65))", 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.356380+0000", 2024-09-17T17:30:53.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.8897228360000007, 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006989337, 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356380+0000", 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356380+0000", 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356381+0000", 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6300000000000005e-07 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356400+0000", 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9814999999999999e-05 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356442+0000", 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2042000000000002e-05 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356452+0000", 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4590000000000004e-06 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356511+0000", 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9218000000000001e-05 2024-09-17T17:30:53.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356524+0000", 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3086e-05 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363325+0000", 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068006719999999998 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363369+0000", 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4382000000000003e-05 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s6 96/74 ECSubWrite(tid=330, reqid=client.4566.0:927, at_version=96'46, trim_to=0'0, roll_forward_to=96'42))", 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.360632+0000", 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8854702840000002, 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080918599999999993, 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360632+0000", 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360632+0000", 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360633+0000", 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0049999999999999e-06 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360781+0000", 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014790199999999999 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360806+0000", 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4391000000000001e-05 2024-09-17T17:30:53.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360813+0000", 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1779999999999997e-06 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360874+0000", 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1099000000000006e-05 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360885+0000", 2024-09-17T17:30:53.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1051e-05 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368683+0000", 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077983890000000002 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368724+0000", 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0844999999999999e-05 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s6 96/74 ECSubWrite(tid=331, reqid=client.4566.0:928, at_version=96'47, trim_to=0'0, roll_forward_to=96'42))", 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.362531+0000", 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8835717250000004, 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0062230469999999998, 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362531+0000", 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362531+0000", 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362532+0000", 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362551+0000", 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9377000000000001e-05 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362586+0000", 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5414999999999999e-05 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362595+0000", 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0269999999999995e-06 2024-09-17T17:30:53.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.363730+0000", 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011350519999999999 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.363738+0000", 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8589999999999994e-06 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368730+0000", 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049914970000000001 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368754+0000", 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4111999999999999e-05 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 100/76 ECSubWrite(tid=237, reqid=client.4566.0:969, at_version=100'75, trim_to=0'0, roll_forward_to=100'71))", 2024-09-17T17:30:53.371 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.473563202, 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075688450000000003, 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2699999999999999e-07 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774069+0000", 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015291079999999999 2024-09-17T17:30:53.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774101+0000", 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1423000000000001e-05 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774115+0000", 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.488e-05 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774190+0000", 2024-09-17T17:30:53.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4799e-05 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774198+0000", 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8110000000000003e-06 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.780079+0000", 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0058809120000000003 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.780108+0000", 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9385000000000001e-05 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.376 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:53.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:30:53.378 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:53.375+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:53.478 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:53.477+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:53.485 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:53.485 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:53.485 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:53.485 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:53.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:30:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T17:30:53.579 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:53.577+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:53.644 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:53.644 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:53.652 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:30:53.679 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:53.678+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:53.760 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 84.177867427999999, 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.27275196200000001, 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580026+0000", 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.359e-06 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580034+0000", 2024-09-17T17:30:53.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580048+0000", 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3974e-05 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580058+0000", 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0064999999999999e-05 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580111+0000", 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3248000000000002e-05 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580119+0000", 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1270000000000003e-06 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847220+0000", 2024-09-17T17:30:53.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26710079599999997 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847357+0000", 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013739500000000001 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847505+0000", 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147188 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.850265+0000", 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002760396 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852701+0000", 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024356059999999999 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852744+0000", 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3095e-05 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852777+0000", 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2784000000000003e-05 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.770379140000003, 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010965536999999999, 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 48 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5899999999999996e-07 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987518+0000", 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0429999999999998e-06 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987523+0000", 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3499999999999999e-06 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987526+0000", 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5159999999999999e-06 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991339+0000", 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038128670000000002 2024-09-17T17:30:53.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991396+0000", 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7219000000000002e-05 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991498+0000", 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010142 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993114+0000", 2024-09-17T17:30:53.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016166900000000001 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998400+0000", 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052850600000000003 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998438+0000", 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8272000000000002e-05 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998478+0000", 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0540999999999997e-05 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.770360789999998, 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011028196000000001, 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 49 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987534+0000", 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4399999999999999e-06 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987538+0000", 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9369999999999997e-06 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987541+0000", 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0350000000000002e-06 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991920+0000", 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043794000000000003 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991972+0000", 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1412000000000002e-05 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992057+0000", 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4917e-05 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993157+0000", 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011003810000000001 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998485+0000", 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053282989999999999 2024-09-17T17:30:53.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998517+0000", 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2019e-05 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998559+0000", 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2076000000000002e-05 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.770339301000007, 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011080325, 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 50 2024-09-17T17:30:53.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-07 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987605+0000", 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2114000000000003e-05 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987631+0000", 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5766999999999999e-05 2024-09-17T17:30:53.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987646+0000", 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515999999999998e-05 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992371+0000", 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047247360000000002 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992433+0000", 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1451000000000005e-05 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992522+0000", 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9621000000000001e-05 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994510+0000", 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001988227 2024-09-17T17:30:53.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998564+0000", 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040533710000000001 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998595+0000", 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0843000000000003e-05 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998633+0000", 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8321000000000001e-05 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.770227276, 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011043702000000001, 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 51 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987669+0000", 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7189999999999999e-06 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987674+0000", 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1780000000000002e-06 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987678+0000", 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5999999999999998e-06 2024-09-17T17:30:53.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993192+0000", 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055137620000000002 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993262+0000", 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993349+0000", 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7362999999999997e-05 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994549+0000", 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011994110000000001 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998640+0000", 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040912409999999998 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998669+0000", 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9209000000000001e-05 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998708+0000", 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9419000000000003e-05 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.794 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.770207185999993, 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011098023, 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 52 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:30:53.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2000000000000001e-07 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987688+0000", 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4729999999999999e-06 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987693+0000", 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5010000000000004e-06 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987698+0000", 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1499999999999998e-06 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994130+0000", 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064314890000000003 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994185+0000", 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5769000000000001e-05 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994272+0000", 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6638000000000006e-05 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994990+0000", 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000717463 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998715+0000", 2024-09-17T17:30:53.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003725 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998743+0000", 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8830999999999999e-05 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998783+0000", 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9388999999999998e-05 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.460492692000003, 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011397086000000001, 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 76 2024-09-17T17:30:53.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297400+0000", 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7100000000000002e-07 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297403+0000", 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5920000000000001e-06 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297410+0000", 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1489999999999997e-06 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297417+0000", 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-06 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306131+0000", 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087138279999999999 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306185+0000", 2024-09-17T17:30:53.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3783999999999999e-05 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306278+0000", 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3587999999999999e-05 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306994+0000", 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071571099999999995 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308729+0000", 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001735489 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308769+0000", 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9270000000000002e-05 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308796+0000", 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7603999999999999e-05 2024-09-17T17:30:53.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:124 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.218431140000007, 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011177351, 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 124 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539467+0000", 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2089999999999996e-06 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539473+0000", 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4690000000000002e-06 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539479+0000", 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545059+0000", 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055806400000000004 2024-09-17T17:30:53.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545104+0000", 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5040999999999998e-05 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545172+0000", 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7249999999999995e-05 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:30:53.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545874+0000", 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070271799999999996 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550512+0000", 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046379530000000002 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550597+0000", 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025999999999995e-05 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550638+0000", 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0771e-05 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.813 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.as2 37/32 ECSubRead(tid=27, to_read={3:5ee91470:::smithi10238019-6:head=20480,4096,0}, subchunks={3:5ee91470:::smithi10238019-6:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.764865388000004, 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.671263006, 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993028+0000", 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0219999999999999e-06 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993040+0000", 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1968e-05 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664125+0000", 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.67108543799999998 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664134+0000", 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8640000000000002e-06 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664204+0000", 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9733999999999993e-05 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664290+0000", 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.598e-05 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 43/35 ECSubWrite(tid=54, reqid=osd.11.0:54, at_version=43'10, trim_to=0'0, roll_forward_to=42'8))", 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.056373436000001, 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.282394969, 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6e-07 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701543+0000", 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4530000000000001e-05 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983000+0000", 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.28145619700000002 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983008+0000", 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6510000000000007e-06 2024-09-17T17:30:53.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983087+0000", 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8623999999999998e-05 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983093+0000", 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6599999999999998e-06 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983867+0000", 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000773962 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983914+0000", 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6084999999999997e-05 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 43/35 ECSubWrite(tid=62, reqid=client.4566.0:316, at_version=43'14, trim_to=0'0, roll_forward_to=43'10))", 2024-09-17T17:30:53.822 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.766926484999999, 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.039163349, 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990966+0000", 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3e-07 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990998+0000", 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1909000000000002e-05 2024-09-17T17:30:53.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029506+0000", 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.038507953999999997 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029512+0000", 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3419999999999996e-06 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029561+0000", 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8930000000000001e-05 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029568+0000", 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2480000000000001e-06 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.030089+0000", 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00052149900000000003 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.030129+0000", 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9737000000000003e-05 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.377460521000003, 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31954210500000002, 2024-09-17T17:30:53.827 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380432+0000", 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8500000000000001e-07 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380446+0000", 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4214e-05 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699334+0000", 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31888741599999998 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699350+0000", 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5913000000000002e-05 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699521+0000", 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000171244 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699530+0000", 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8079999999999994e-06 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699939+0000", 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040906299999999998 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699974+0000", 2024-09-17T17:30:53.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4761999999999997e-05 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.18s0 68/67 ECSubWriteReply(tid=153, last_complete=68'25, committed=1, applied=1))", 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.293963333000001, 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.73874762000000005, 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.832 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:30:53.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0400000000000002e-07 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463933+0000", 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9689999999999996e-06 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202459+0000", 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.738526142 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202472+0000", 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2989999999999999e-05 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202624+0000", 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015171499999999999 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202676+0000", 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.24e-05 2024-09-17T17:30:53.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.281901773000001, 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.37910365400000001, 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475991+0000", 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476005+0000", 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.449e-05 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854381+0000", 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.37837595699999998 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854387+0000", 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4689999999999996e-06 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854441+0000", 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4593999999999999e-05 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854449+0000", 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0029999999999994e-06 2024-09-17T17:30:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.855065+0000", 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061543700000000004 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.855094+0000", 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9193999999999999e-05 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:799 3.18s0 3:1f25649f:::smithi10238019-40:head [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339002+0000", 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.418890261000001, 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012582919, 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 799 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.841 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339002+0000", 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339002+0000", 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339005+0000", 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3139999999999999e-06 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339032+0000", 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6529999999999998e-05 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339056+0000", 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.421e-05 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339069+0000", 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339161+0000", 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2238999999999996e-05 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339171+0000", 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9089999999999995e-06 2024-09-17T17:30:53.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348303+0000", 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0091322829999999997 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348416+0000", 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112928 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.351585+0000", 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031685929999999999 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:800 3.18s0 3:1f25649f:::smithi10238019-40:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339103+0000", 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.418789167, 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013185367999999999, 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 800 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339103+0000", 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339103+0000", 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339104+0000", 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1100000000000005e-07 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339131+0000", 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7180999999999999e-05 2024-09-17T17:30:53.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339211+0000", 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9802000000000006e-05 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339221+0000", 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0277e-05 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339434+0000", 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0002131 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339442+0000", 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6620000000000005e-06 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348818+0000", 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093763130000000007 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348904+0000", 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6069000000000006e-05 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352288+0000", 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003384153 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:801 3.18s0 3:1f25649f:::smithi10238019-40:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339263+0000", 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.418629166999999, 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.98368640699999998, 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 801 2024-09-17T17:30:53.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339263+0000", 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339263+0000", 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339264+0000", 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1850000000000001e-06 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339271+0000", 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7299999999999999e-06 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339281+0000", 2024-09-17T17:30:53.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0674e-05 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339290+0000", 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3450000000000006e-06 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339467+0000", 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000177636 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339473+0000", 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7409999999999998e-06 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.349129+0000", 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.009656296 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.349185+0000", 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6057999999999998e-05 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322949+0000", 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.97376374200000004 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.857 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.18s0 80/75 ECSubReadReply(tid=169, attrs_read=0))", 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.407107429, 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.97217706000000004, 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8000000000000001e-07 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350860+0000", 2024-09-17T17:30:53.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5136999999999996e-05 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322452+0000", 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.97159191899999997 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322462+0000", 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9559999999999992e-06 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322532+0000", 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0222999999999995e-05 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322962+0000", 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00042944499999999998 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.861 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:964 3.18s0 3:1f25649f:::smithi10238019-40:head [copy-from ver 60] snapc 22=[22,21,20,18,17,10] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:51.753402+0000", 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.0044899859999998, 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.042342484, 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 964 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753402+0000", 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753402+0000", 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753414+0000", 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2306999999999999e-05 2024-09-17T17:30:53.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753449+0000", 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5151999999999998e-05 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753481+0000", 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1702999999999998e-05 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753489+0000", 2024-09-17T17:30:53.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.729e-06 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753555+0000", 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6277000000000003e-05 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753640+0000", 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5048000000000003e-05 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.787721+0000", 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.034080810000000003 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.791433+0000", 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037123429999999999 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.795661+0000", 2024-09-17T17:30:53.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042272840000000004 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.795744+0000", 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3831000000000004e-05 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:965 3.18s0 3:1f25649f:::smithi10238019-40:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:30:53.867 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:51.753508+0000", 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.0043836189999999, 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.037400758999999999, 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 965 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753508+0000", 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753508+0000", 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753509+0000", 2024-09-17T17:30:53.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8100000000000002e-07 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753515+0000", 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3910000000000003e-06 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753525+0000", 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4320000000000005e-06 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753529+0000", 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6430000000000003e-06 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753787+0000", 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025746599999999998 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753845+0000", 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7667999999999997e-05 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.790726+0000", 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.036881635000000003 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.790817+0000", 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1205000000000005e-05 2024-09-17T17:30:53.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.790909+0000", 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1638000000000006e-05 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:53.873 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:53.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:30:53.875 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:53.873+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:53.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:53.989 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:53.987+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:54.022 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:54.022 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:54.022 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:54.023 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:54.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:30:54.076 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:54.074+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:54.137 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:54.137 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:54.137 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:30:54.137 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:30:54.137 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:54.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:30:54.177 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:54.175+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:54.223+0000 7f0a237fe640 0 --2- 172.21.15.7:0/1101540629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a24158ce0 0x7f0a241f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T17:30:54.252 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:54.252 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:54.253 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:218 3.3s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.529209844999997, 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65573700899999998, 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 218 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721616+0000", 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4e-07 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721619+0000", 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2030000000000001e-06 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721626+0000", 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721631+0000", 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4539999999999998e-06 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722624+0000", 2024-09-17T17:30:54.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00099297500000000011 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722675+0000", 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0627000000000002e-05 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722943+0000", 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026790700000000001 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723463+0000", 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000520003 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377270+0000", 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65380684099999997 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377352+0000", 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2307999999999999e-05 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:219 3.3s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.529183621000001, 2024-09-17T17:30:54.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65649164999999998, 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 219 2024-09-17T17:30:54.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721642+0000", 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0100000000000005e-07 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721644+0000", 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4719999999999998e-06 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721651+0000", 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.781e-06 2024-09-17T17:30:54.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721656+0000", 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4809999999999997e-06 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723147+0000", 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001490587 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:54.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723197+0000", 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.003e-05 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377653+0000", 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65445650300000002 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377716+0000", 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2223000000000004e-05 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.378133+0000", 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041707199999999999 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:440 3.3s0 3:c80ea1ca:::smithi10238019-27:head [append 0~81920] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.846450099000002, 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19940718800000001, 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 440 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8299999999999996e-07 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404418+0000", 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2608000000000003e-05 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404437+0000", 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8810000000000001e-05 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404442+0000", 2024-09-17T17:30:54.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.327e-06 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406246+0000", 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018034769999999999 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406352+0000", 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106269 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406784+0000", 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00043238800000000002 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.408111+0000", 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001326502 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603719+0000", 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.195608474 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603782+0000", 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2550000000000003e-05 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/35 ECSubWriteReply(tid=48, last_complete=52'36, committed=1, applied=1))", 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.843054961, 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19602166600000001, 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407771+0000", 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1100000000000004e-07 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407775+0000", 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5510000000000001e-06 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603542+0000", 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19576712299999999 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603557+0000", 2024-09-17T17:30:54.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4229999999999999e-05 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603658+0000", 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010186100000000001 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603791+0000", 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013299000000000001 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.as3 62/32 ECSubWrite(tid=104, reqid=client.4566.0:558, at_version=62'84, trim_to=0'0, roll_forward_to=62'79))", 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.748889927, 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95386532199999996, 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501936+0000", 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0109999999999999e-06 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501948+0000", 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2544000000000001e-05 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454818+0000", 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95286919199999998 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454827+0000", 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3130000000000004e-06 2024-09-17T17:30:54.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454959+0000", 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013207099999999999 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454969+0000", 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0368e-05 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455761+0000", 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000792051 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455800+0000", 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8772000000000001e-05 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=123, reqid=client.4566.0:616, at_version=0'0, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.801012370999999, 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015260055, 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449813+0000", 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449823+0000", 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9539999999999999e-06 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464497+0000", 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014673699 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464503+0000", 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1770000000000003e-06 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464557+0000", 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:30:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464568+0000", 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0386e-05 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465032+0000", 2024-09-17T17:30:54.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046443199999999997 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465072+0000", 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0280000000000001e-05 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=125, reqid=client.4566.0:619, at_version=68'49, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.784488948000003, 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010346985, 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466642+0000", 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030544600000000002 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475441+0000", 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087995179999999992 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475448+0000", 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4439999999999996e-06 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475494+0000", 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000003e-05 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475500+0000", 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476652+0000", 2024-09-17T17:30:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001152887 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476683+0000", 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0306000000000001e-05 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:30:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.774752847999999, 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86543154899999997, 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476073+0000", 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4900000000000005e-07 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476093+0000", 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0431000000000001e-05 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340542+0000", 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86444875899999996 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340554+0000", 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2017e-05 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340635+0000", 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1500000000000002e-05 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340651+0000", 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5444999999999999e-05 2024-09-17T17:30:54.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341467+0000", 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00081634899999999996 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341503+0000", 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6298999999999999e-05 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.299 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:644 3.3s0 3:c77cc1f5:::smithi10238019-3:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.551731208, 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.524887887, 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.300 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 644 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699095+0000", 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.099e-06 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699100+0000", 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1549999999999998e-06 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699109+0000", 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1670000000000005e-06 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699117+0000", 2024-09-17T17:30:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8720000000000002e-06 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699171+0000", 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4237999999999998e-05 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699278+0000", 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107339 2024-09-17T17:30:54.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699941+0000", 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066276400000000004 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.702383+0000", 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024413550000000001 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223916+0000", 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52153379200000005 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223981+0000", 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5105999999999996e-05 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 69/67 ECSubWrite(tid=129, reqid=client.4566.0:621, at_version=0'0, trim_to=0'0, roll_forward_to=68'51))", 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 33.908036838000001, 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014148114, 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342789+0000", 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.13e-06 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342803+0000", 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4309e-05 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356208+0000", 2024-09-17T17:30:54.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013404337000000001 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356219+0000", 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1569000000000001e-05 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356293+0000", 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3894999999999996e-05 2024-09-17T17:30:54.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356301+0000", 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5159999999999997e-06 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356892+0000", 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059179499999999999 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356936+0000", 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3563000000000003e-05 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:725 3.3s0 3:c5fa5218:::smithi10238019-30:head [create,setxattr n\\kYkdan\\thNN\\sTap[lWfmsQQctdU^aYUdXcPaN (278) in=40b,setxattr gdposbmqnPajgmVYW]VdX\\`gUQkaeU]UtbYUfiNg (1750) in=40b,setxattr ciVa^RQRQUnlPcOS[RhXiP[f`fVknjrV_`VY^mUX (417) in=40b,setxattr d_hhtYW (11296) in=7b,setxattr sRYR``_hO_QkVYhhONZZjoiUtaoqebe^l]rOq[_Y (1682) in=40b,setxattr _QNRabNbNqfbhqjYjpV`OpcjWYck]PSYrhXT_pR` (350) in=40b,setxattr Orajpk`Qfg^nQasaZW_ePrgt^]N__bT[NVS]PaaW (19224) in=40b,setxattr e[aOkpoq_NTqbYdOapYpT (11296) in=21b,setxattr ogiNjhNa]OiT]hhXbR_ti`jYmsgbs_rtq_jR`[`g (979) in=40b,setxattr j (11296) in=1b,setxattr N\\i[nSgaepe]]Wrqk[^`enQo]SlraVR^o_SO (11296) in=36b,setxattr cUpO`i`Te\\O\\Wtmgsdpghq]jam\\WhaNdZTYgepZZ (1049) in=40b,setxattr tU[pQ\\_U_siq^t[VT_bha\\Qjp`Y_n``SZlnPr`Nj (1329) in=40b,setxattr trgNslOknYrnhc`^OllRjkWjZc (11296) in=26b,setxattr ofS\\dVZbbra]Vf^gm[[Y^^XW`gWa_PjOdRlafjTa (66) in=40b,setxattr k^Ue`eUT^[jiTlg\\aPXs`ZXXaejRaY^X^\\oQ_clj (1262) in=40b,setxattr _goNc]^eaWViaTbdiVPjTWXpe_VrQgcY]nSajfQk (242) in=40b,setxattr sfdRTqXeR]_gb]lnNOUWXWfc\\NQcbOaTrTk`aOQq (700) in=40b,setxattr Yrh^hjspm\\YXTjTb]TRjpnsQU]sq\\c`RcfSeOchd (7436) in=40b,setxattr aahttaRSQ\\X`]orTWZRe[_OplTjd`YRoTQdeTmmj (1926) in=40b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 28.913160690000002, 2024-09-17T17:30:54.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0205677150000001, 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 725 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337666+0000", 2024-09-17T17:30:54.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.513e-06 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337795+0000", 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000128961 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337808+0000", 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3713000000000001e-05 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337819+0000", 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.027e-05 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341249+0000", 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034306190000000002 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341253+0000", 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4410000000000002e-06 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.346358+0000", 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.005105844 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.346550+0000", 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019203699999999999 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.347606+0000", 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001055896 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.353654+0000", 2024-09-17T17:30:54.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060472520000000004 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358123+0000", 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044693620000000002 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358232+0000", 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108807 2024-09-17T17:30:54.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 75/74 ECSubWriteReply(tid=94, last_complete=75'58, committed=1, applied=1))", 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.895680941999998, 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.74451308800000005, 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355148+0000", 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-06 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098760+0000", 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.74361193699999995 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098773+0000", 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.288e-05 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098921+0000", 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014810999999999999 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.099657+0000", 2024-09-17T17:30:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00073597300000000001 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:784 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~720896] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.899352406999999, 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010356648, 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 784 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:30:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351474+0000", 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.562e-06 2024-09-17T17:30:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352295+0000", 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00082097799999999999 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352310+0000", 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5599000000000001e-05 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352317+0000", 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0900000000000001e-06 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352366+0000", 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9768999999999997e-05 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352456+0000", 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0044999999999996e-05 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353483+0000", 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001026788 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357386+0000", 2024-09-17T17:30:54.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039030559999999998 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361725+0000", 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043392559999999997 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361829+0000", 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010350500000000001 2024-09-17T17:30:54.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.332 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:785 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~622592] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.895782780000001, 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010812743, 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 785 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355044+0000", 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1919999999999999e-06 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355710+0000", 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066595000000000001 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355726+0000", 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6030999999999999e-05 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355735+0000", 2024-09-17T17:30:54.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7390000000000005e-06 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355799+0000", 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4281000000000003e-05 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355877+0000", 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7553999999999996e-05 2024-09-17T17:30:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.356754+0000", 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087738099999999999 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361867+0000", 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0051130380000000003 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365763+0000", 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003895496 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365855+0000", 2024-09-17T17:30:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2081000000000001e-05 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:788 3.13s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (57) in=7b] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.889906195000002, 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014979750999999999, 2024-09-17T17:30:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 788 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4000000000000002e-07 2024-09-17T17:30:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360925+0000", 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4010000000000002e-06 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360932+0000", 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1879999999999999e-06 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360937+0000", 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4330000000000003e-06 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.364485+0000", 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035482949999999999 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.364590+0000", 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104424 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365073+0000", 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048325000000000001 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367135+0000", 2024-09-17T17:30:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002061677 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375816+0000", 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086812480000000008 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375898+0000", 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2294999999999997e-05 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:789 3.13s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.889879384, 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016800492, 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 789 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360946+0000", 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4300000000000003e-07 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360950+0000", 2024-09-17T17:30:54.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.25e-06 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360958+0000", 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5360000000000004e-06 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360963+0000", 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5509999999999999e-06 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366330+0000", 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053669410000000001 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366434+0000", 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103384 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.376442+0000", 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010008349 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.376609+0000", 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000166732 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377746+0000", 2024-09-17T17:30:54.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001137206 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.12s5 85/84 ECSubRead(tid=64, to_read={3:4c833192:::smithi10238019-39:head=0,262144,0}, subchunks={3:4c833192:::smithi10238019-39:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.870253422000001, 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.043000999999999998, 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380573+0000", 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.576e-06 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380584+0000", 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1195e-05 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422791+0000", 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.042206670000000002 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422808+0000", 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7677e-05 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422941+0000", 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013300900000000001 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.423572+0000", 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000630873 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:925 3.3s0 3:c0b80cc2:::smithi10238019-19:head [copy-from ver 153] snapc 20=[20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.345038+0000", 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.9057872190000005, 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.041407069999999997, 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 925 2024-09-17T17:30:54.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345038+0000", 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345038+0000", 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345039+0000", 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.486e-06 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345049+0000", 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0370000000000001e-05 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345074+0000", 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4453000000000001e-05 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345083+0000", 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5859999999999994e-06 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345145+0000", 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1983000000000006e-05 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345160+0000", 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4321000000000001e-05 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.352926+0000", 2024-09-17T17:30:54.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077658359999999999 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.353042+0000", 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011646699999999999 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.380319+0000", 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027276584 2024-09-17T17:30:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.384772+0000", 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004453785 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.386372+0000", 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016000960000000001 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.386445+0000", 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2102999999999997e-05 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:926 3.3s0 3:c0b80cc2:::smithi10238019-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.345110+0000", 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.9057145310000001, 2024-09-17T17:30:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.037529039, 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 926 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345110+0000", 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345110+0000", 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345111+0000", 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1299999999999999e-07 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345115+0000", 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7689999999999998e-06 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345123+0000", 2024-09-17T17:30:54.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-17T17:30:54.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345130+0000", 2024-09-17T17:30:54.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8759999999999999e-06 2024-09-17T17:30:54.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345224+0000", 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4067000000000006e-05 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345232+0000", 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3939999999999998e-06 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.353253+0000", 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080215349999999998 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.353329+0000", 2024-09-17T17:30:54.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5767999999999997e-05 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.382446+0000", 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.029117214999999998 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.382536+0000", 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9852000000000006e-05 2024-09-17T17:30:54.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.382639+0000", 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010304799999999999 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:934 3.3s0 3:c5fa5218:::smithi10238019-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:48.360510+0000", 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8903142879999999, 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012112242, 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:30:54.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 934 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360510+0000", 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360510+0000", 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360511+0000", 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.37e-07 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360514+0000", 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3790000000000002e-06 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360522+0000", 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9160000000000001e-06 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360528+0000", 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7100000000000004e-06 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360619+0000", 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1333000000000001e-05 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360623+0000", 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8850000000000001e-06 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.365886+0000", 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052631880000000002 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:54.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.365996+0000", 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109456 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.370760+0000", 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047638439999999997 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.370864+0000", 2024-09-17T17:30:54.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103968 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.372623+0000", 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0017589260000000001 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:30:54.381 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:54.383 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:54.384 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:54.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:30:54.385 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:54.381+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:54.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:54.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:54.405 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:54.405 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:54.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:54.447 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:54.447 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:54.467 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:54.468 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:54.484 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:54.483+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:54.490 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:54.510 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:54.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:54.530+0000 7f0a23fff640 0 -- 172.21.15.7:0/1101540629 >> [v2:172.21.15.26:6800/1983765258,v1:172.21.15.26:6801/1983765258] conn(0x7f0a24214e40 msgr2=0x7f0a24217230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:54.532 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:30:54.532 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:30:54.533 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:30:54.533 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:30:54.538 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:54.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:30:54.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:54.585 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:54.583+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:54.586 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:54.686 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:54.683+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:54.688 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:54.688 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:54.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:30:54.786 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:54.784+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:54.803 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:54.804 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:54.812 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:30:54.887 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:54.885+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:54.924 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:54.924 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:54.933 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:30:54.988 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:54.986+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:55.036 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.036 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.045 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:30:55.089 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:55.087+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:55.148 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.149 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:30:55.190 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:55.188+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:55.291 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:55.289+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:55.301 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.301 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.310 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:30:55.392 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:55.391+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:55.417 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.418 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.426 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:30:55.493 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:55.492+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:55.529 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.529 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.538 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:30:55.548 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 2] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:30:55.548 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:30:55.548 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T17:30:55.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T17:30:55.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T17:30:55.594 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:55.592+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:55.641 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.641 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.649 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:30:55.695 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:55.693+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:55.752 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.752 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:30:55.795 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:55.793+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:55.864 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:55.864 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:55.872 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:30:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 104, 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "2593b77f-a53d-4649-b5a5-0122bd31e2b3", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T17:29:15.547925+0000", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T17:30:55.337756+0000", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T17:29:28.558396+0000", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T17:30:22.332815+0000", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5801984, 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-17T17:30:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 7, 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 16, 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T17:30:55.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-17T17:30:55.888 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T17:30:55.888 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T17:30:55.888 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T17:30:55.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.888 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T17:30:55.888 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T17:29:27.893486+0000", 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T17:30:55.889 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T17:30:55.890 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "25", 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T17:30:55.891 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T17:30:55.892 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T17:30:55.893 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T17:30:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 15.380000114440918, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 15.380000114440918, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T17:30:55.895 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T17:29:30.150043+0000", 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T17:30:55.896 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T17:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "28", 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T17:30:55.898 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 28, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T17:30:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T17:30:55.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T17:30:55.901 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 4.0199999809265137, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 4.0199999809265137, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.56000000238418579, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T17:30:55.902 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "unique_pool_0", 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T17:29:42.633017+0000", 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 3, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 7, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 5, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 1, 2024-09-17T17:30:55.903 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 26, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 22, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 26, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 26, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 26, 2024-09-17T17:30:55.904 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "67", 2024-09-17T17:30:55.905 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 34, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 100, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T17:30:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "shecprofile", 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T17:30:55.907 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 16384, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: "rados": {} 2024-09-17T17:30:55.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "72ee94e9-57bf-46b3-9fb9-0d05515fe1e1", 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.909 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 103, 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 50344815 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/50344815", 2024-09-17T17:30:55.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/50344815", 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/50344815", 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/50344815", 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e9764092-a458-4b6d-8bee-658f54f000cc", 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.68646240234375, 2024-09-17T17:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 21, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 94, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6816", 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6817", 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6818", 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6819", 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6822", 2024-09-17T17:30:55.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6823", 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6820", 2024-09-17T17:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6821", 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2426784045 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6817/2426784045", 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6819/2426784045", 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6823/2426784045", 2024-09-17T17:30:55.920 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6821/2426784045", 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f1142f7d-9945-4daa-8f6a-6fd2d3090b5f", 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.921 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 0, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6800", 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6801", 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6802", 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6803", 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6806", 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6807", 2024-09-17T17:30:55.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6804", 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6805", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1502510538 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6801/1502510538", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6803/1502510538", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6807/1502510538", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6805/1502510538", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "daa4b149-c71a-4cba-a710-89aa101d7d96", 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 102, 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6810", 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6811", 2024-09-17T17:30:55.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6812", 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6813", 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6816", 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6817", 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6814", 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6815", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2453627254 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6811/2453627254", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6813/2453627254", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6817/2453627254", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6815/2453627254", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "55e8e034-493b-4066-a42b-f4d82294f3f8", 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 84, 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T17:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T17:30:55.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3507123258 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/3507123258", 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/3507123258", 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/3507123258", 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/3507123258", 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "38a09763-d370-447a-b939-bad8a11c5e01", 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 0, 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6824", 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6825", 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6826", 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6827", 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6830", 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6831", 2024-09-17T17:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6828", 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6829", 2024-09-17T17:30:55.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 229014119 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6825/229014119", 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6827/229014119", 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6831/229014119", 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6829/229014119", 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "0ae069f9-8831-4bfc-9928-2100b73a839d", 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.946 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6816", 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6817", 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6818", 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6819", 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6822", 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6823", 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6820", 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6821", 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3536708597 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6817/3536708597", 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6819/3536708597", 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6823/3536708597", 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6821/3536708597", 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b", 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 21, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 88, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.952 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6802", 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6803", 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6804", 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6805", 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6808", 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6809", 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6806", 2024-09-17T17:30:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6807", 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2367087286 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6803/2367087286", 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6805/2367087286", 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6809/2367087286", 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6807/2367087286", 2024-09-17T17:30:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 8, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "3c4daae0-6d93-450f-8600-b29362bd9f77", 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:55.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 32, 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T17:30:55.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T17:30:55.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T17:30:55.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1273991967 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/1273991967", 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/1273991967", 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1273991967", 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1273991967", 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 9, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2846b3c6-06d6-414f-b4c9-b16d94007737", 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 25, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6808", 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6809", 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6810", 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6811", 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6814", 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6815", 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6812", 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6813", 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2309011334 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6809/2309011334", 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6811/2309011334", 2024-09-17T17:30:55.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6815/2309011334", 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6813/2309011334", 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 10, 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "495d7d7c-c222-4a46-9685-2a2a7cc6fc51", 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.972 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 102, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6808", 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6809", 2024-09-17T17:30:55.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6810", 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6811", 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6814", 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6815", 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6812", 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6813", 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2168900921 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6809/2168900921", 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6811/2168900921", 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6815/2168900921", 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6813/2168900921", 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 11, 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf76d383-65cd-48c3-a54c-cd83ffbc9898", 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6826", 2024-09-17T17:30:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6827", 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6828", 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6829", 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6832", 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6833", 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6830", 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6831", 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3109125505 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6827/3109125505", 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6829/3109125505", 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6833/3109125505", 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6831/3109125505", 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 12, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "54294878-ea1e-4845-a0e6-9be026f521fa", 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.74237060546875, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 23, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 76, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.987 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-17T17:30:55.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-17T17:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.991 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-17T17:30:55.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3659919075 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/3659919075", 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/3659919075", 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/3659919075", 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/3659919075", 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 13, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "9acf83a5-75db-4874-8f3f-7421cc55e2fd", 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.8008880615234375, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:55.994 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 95, 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6800", 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6801", 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6802", 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6803", 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6806", 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6807", 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:55.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6804", 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.26:6805", 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1983765258 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.26:6801/1983765258", 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.26:6803/1983765258", 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.26:6807/1983765258", 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.26:6805/1983765258", 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 14, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "16c02ebe-a418-4f87-902d-b5dac864f139", 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:56.001 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 101, 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6824", 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6825", 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6826", 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6827", 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6830", 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6831", 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:56.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6828", 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.27:6829", 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1199123829 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.27:6825/1199123829", 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.27:6827/1199123829", 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.27:6831/1199123829", 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.27:6829/1199123829", 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 15, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa", 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 22, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 94, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T17:30:56.008 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6818", 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6819", 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6820", 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6821", 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6824", 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6825", 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6822", 2024-09-17T17:30:56.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.102:6823", 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 259571866 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.102:6819/259571866", 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.102:6821/259571866", 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.102:6825/259571866", 2024-09-17T17:30:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.102:6823/259571866", 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:53.149089+0000", 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:27.557824+0000", 2024-09-17T17:30:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:50.042615+0000", 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:44.107535+0000", 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:43.930932+0000", 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:46.730260+0000", 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-17T17:30:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:43.862104+0000", 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-17T17:30:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:42.595312+0000", 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 8, 2024-09-17T17:30:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:40.607805+0000", 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 9, 2024-09-17T17:30:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:40.265380+0000", 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 10, 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:33.448841+0000", 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 11, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:29.957316+0000", 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 12, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:29.888647+0000", 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 13, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:48.511757+0000", 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 14, 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:48.711104+0000", 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 15, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T17:30:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T17:30:32.994404+0000", 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.5", 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: 10, 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-09-17T17:30:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: 14, 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T17:30:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "shecprofile": { 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "c": "2", 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-device-class": "", 2024-09-17T17:30:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-num-failure-domains": "0", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-osds-per-failure-domain": "0", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-root": "default", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "4", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "3", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "shecprofile", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "shec", 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "multiple" 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 21, 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 2 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [ 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 19, 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 25, 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 7 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:56.035 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:56.038 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.038 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.039 INFO:tasks.thrashosds.thrasher:j is {'epoch': 104, 'fsid': '2593b77f-a53d-4649-b5a5-0122bd31e2b3', 'created': '2024-09-17T17:29:15.547925+0000', 'modified': '2024-09-17T17:30:55.337756+0000', 'last_up_change': '2024-09-17T17:29:28.558396+0000', 'last_in_change': '2024-09-17T17:30:22.332815+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T17:29:27.893486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T17:29:30.150043+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-17T17:29:42.633017+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 22, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '67', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 34, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '72ee94e9-57bf-46b3-9fb9-0d05515fe1e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 50344815}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 50344815}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 50344815}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 50344815}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 50344815}]}, 'public_addr': '172.21.15.7:6817/50344815', 'cluster_addr': '172.21.15.7:6819/50344815', 'heartbeat_back_addr': '172.21.15.7:6823/50344815', 'heartbeat_front_addr': '172.21.15.7:6821/50344815', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e9764092-a458-4b6d-8bee-658f54f000cc', 'up': 1, 'in': 1, 'weight': 0.68646240234375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6816', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6817', 'nonce': 2426784045}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6818', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6819', 'nonce': 2426784045}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6822', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6823', 'nonce': 2426784045}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6820', 'nonce': 2426784045}, {'type': 'v1', 'addr': '172.21.15.26:6821', 'nonce': 2426784045}]}, 'public_addr': '172.21.15.26:6817/2426784045', 'cluster_addr': '172.21.15.26:6819/2426784045', 'heartbeat_back_addr': '172.21.15.26:6823/2426784045', 'heartbeat_front_addr': '172.21.15.26:6821/2426784045', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f1142f7d-9945-4daa-8f6a-6fd2d3090b5f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 1502510538}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 1502510538}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 1502510538}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 1502510538}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 1502510538}]}, 'public_addr': '172.21.15.27:6801/1502510538', 'cluster_addr': '172.21.15.27:6803/1502510538', 'heartbeat_back_addr': '172.21.15.27:6807/1502510538', 'heartbeat_front_addr': '172.21.15.27:6805/1502510538', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'daa4b149-c71a-4cba-a710-89aa101d7d96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6810', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6811', 'nonce': 2453627254}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6812', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6813', 'nonce': 2453627254}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6816', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 2453627254}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6814', 'nonce': 2453627254}, {'type': 'v1', 'addr': '172.21.15.102:6815', 'nonce': 2453627254}]}, 'public_addr': '172.21.15.102:6811/2453627254', 'cluster_addr': '172.21.15.102:6813/2453627254', 'heartbeat_back_addr': '172.21.15.102:6817/2453627254', 'heartbeat_front_addr': '172.21.15.102:6815/2453627254', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '55e8e034-493b-4066-a42b-f4d82294f3f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 3507123258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 3507123258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 3507123258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 3507123258}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 3507123258}]}, 'public_addr': '172.21.15.7:6801/3507123258', 'cluster_addr': '172.21.15.7:6803/3507123258', 'heartbeat_back_addr': '172.21.15.7:6807/3507123258', 'heartbeat_front_addr': '172.21.15.7:6805/3507123258', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '38a09763-d370-447a-b939-bad8a11c5e01', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6824', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6825', 'nonce': 229014119}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6826', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6827', 'nonce': 229014119}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6830', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6831', 'nonce': 229014119}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6828', 'nonce': 229014119}, {'type': 'v1', 'addr': '172.21.15.26:6829', 'nonce': 229014119}]}, 'public_addr': '172.21.15.26:6825/229014119', 'cluster_addr': '172.21.15.26:6827/229014119', 'heartbeat_back_addr': '172.21.15.26:6831/229014119', 'heartbeat_front_addr': '172.21.15.26:6829/229014119', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '0ae069f9-8831-4bfc-9928-2100b73a839d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 3536708597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 3536708597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 3536708597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 3536708597}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 3536708597}]}, 'public_addr': '172.21.15.27:6817/3536708597', 'cluster_addr': '172.21.15.27:6819/3536708597', 'heartbeat_back_addr': '172.21.15.27:6823/3536708597', 'heartbeat_front_addr': '172.21.15.27:6821/3536708597', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6802', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6803', 'nonce': 2367087286}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6804', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6805', 'nonce': 2367087286}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6808', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6809', 'nonce': 2367087286}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6806', 'nonce': 2367087286}, {'type': 'v1', 'addr': '172.21.15.102:6807', 'nonce': 2367087286}]}, 'public_addr': '172.21.15.102:6803/2367087286', 'cluster_addr': '172.21.15.102:6805/2367087286', 'heartbeat_back_addr': '172.21.15.102:6809/2367087286', 'heartbeat_front_addr': '172.21.15.102:6807/2367087286', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '3c4daae0-6d93-450f-8600-b29362bd9f77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 1273991967}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1273991967}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1273991967}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1273991967}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1273991967}]}, 'public_addr': '172.21.15.7:6809/1273991967', 'cluster_addr': '172.21.15.7:6811/1273991967', 'heartbeat_back_addr': '172.21.15.7:6815/1273991967', 'heartbeat_front_addr': '172.21.15.7:6813/1273991967', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '2846b3c6-06d6-414f-b4c9-b16d94007737', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6808', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6809', 'nonce': 2309011334}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6810', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6811', 'nonce': 2309011334}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6814', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6815', 'nonce': 2309011334}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6812', 'nonce': 2309011334}, {'type': 'v1', 'addr': '172.21.15.26:6813', 'nonce': 2309011334}]}, 'public_addr': '172.21.15.26:6809/2309011334', 'cluster_addr': '172.21.15.26:6811/2309011334', 'heartbeat_back_addr': '172.21.15.26:6815/2309011334', 'heartbeat_front_addr': '172.21.15.26:6813/2309011334', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '495d7d7c-c222-4a46-9685-2a2a7cc6fc51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 2168900921}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 2168900921}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 2168900921}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 2168900921}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 2168900921}]}, 'public_addr': '172.21.15.27:6809/2168900921', 'cluster_addr': '172.21.15.27:6811/2168900921', 'heartbeat_back_addr': '172.21.15.27:6815/2168900921', 'heartbeat_front_addr': '172.21.15.27:6813/2168900921', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'cf76d383-65cd-48c3-a54c-cd83ffbc9898', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6826', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6827', 'nonce': 3109125505}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6828', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6829', 'nonce': 3109125505}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 3109125505}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6830', 'nonce': 3109125505}, {'type': 'v1', 'addr': '172.21.15.102:6831', 'nonce': 3109125505}]}, 'public_addr': '172.21.15.102:6827/3109125505', 'cluster_addr': '172.21.15.102:6829/3109125505', 'heartbeat_back_addr': '172.21.15.102:6833/3109125505', 'heartbeat_front_addr': '172.21.15.102:6831/3109125505', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '54294878-ea1e-4845-a0e6-9be026f521fa', 'up': 1, 'in': 1, 'weight': 0.74237060546875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 3659919075}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 3659919075}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 3659919075}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 3659919075}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 3659919075}]}, 'public_addr': '172.21.15.7:6825/3659919075', 'cluster_addr': '172.21.15.7:6827/3659919075', 'heartbeat_back_addr': '172.21.15.7:6831/3659919075', 'heartbeat_front_addr': '172.21.15.7:6829/3659919075', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '9acf83a5-75db-4874-8f3f-7421cc55e2fd', 'up': 1, 'in': 1, 'weight': 0.8008880615234375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6800', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6801', 'nonce': 1983765258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6802', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6803', 'nonce': 1983765258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6806', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6807', 'nonce': 1983765258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6804', 'nonce': 1983765258}, {'type': 'v1', 'addr': '172.21.15.26:6805', 'nonce': 1983765258}]}, 'public_addr': '172.21.15.26:6801/1983765258', 'cluster_addr': '172.21.15.26:6803/1983765258', 'heartbeat_back_addr': '172.21.15.26:6807/1983765258', 'heartbeat_front_addr': '172.21.15.26:6805/1983765258', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '16c02ebe-a418-4f87-902d-b5dac864f139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6824', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6825', 'nonce': 1199123829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6826', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6827', 'nonce': 1199123829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6830', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6831', 'nonce': 1199123829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6828', 'nonce': 1199123829}, {'type': 'v1', 'addr': '172.21.15.27:6829', 'nonce': 1199123829}]}, 'public_addr': '172.21.15.27:6825/1199123829', 'cluster_addr': '172.21.15.27:6827/1199123829', 'heartbeat_back_addr': '172.21.15.27:6831/1199123829', 'heartbeat_front_addr': '172.21.15.27:6829/1199123829', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': 'a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6818', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6819', 'nonce': 259571866}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6820', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6821', 'nonce': 259571866}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6824', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6825', 'nonce': 259571866}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6822', 'nonce': 259571866}, {'type': 'v1', 'addr': '172.21.15.102:6823', 'nonce': 259571866}]}, 'public_addr': '172.21.15.102:6819/259571866', 'cluster_addr': '172.21.15.102:6821/259571866', 'heartbeat_back_addr': '172.21.15.102:6825/259571866', 'heartbeat_front_addr': '172.21.15.102:6823/259571866', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:53.149089+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:27.557824+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:50.042615+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:44.107535+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:43.930932+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:46.730260+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:43.862104+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:42.595312+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:40.607805+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:40.265380+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:33.448841+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:29.957316+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:29.888647+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:48.511757+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:48.711104+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T17:30:32.994404+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.5', 'osds': [10, 0, 5, 2, 7, 14, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 19, 'length': 1}, {'begin': 25, 'length': 7}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T17:30:56.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:30:56.040 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:30:56.041 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:56.035+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:56.138 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:56.136+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:56.144 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.144 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.153 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:30:56.238 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:30:56.237+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:30:56.254 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.254 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:30:56.340 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:56.338+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T17:30:56.376 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.376 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:30:56.387 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:30:56.387 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:30:56.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-17T17:30:56.341575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":138576435,"num_objects":119,"num_object_clones":65,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":17,"num_read":320,"num_read_kb":78145,"num_write":655,"num_write_kb":98642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":81362944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1957,"ondisk_log_size":1957,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":807320,"kb_used_data":273556,"kb_used_omap":24,"kb_used_meta":533351,"kb_avail":1509142120,"statfs":{"total":1546188226560,"available":1545361530880,"internally_reserved":0,"allocated":280121344,"data_stored":385899035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":546151503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9076736,"num_objects":8,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":24,"num_read_kb":15660,"num_write":36,"num_write_kb":11184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.800487"},"pg_stats":[{"pgid":"3.19","version":"65'44","reported_seq":408,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548564+0000","last_change":"2024-09-17T17:30:51.537374+0000","last_active":"2024-09-17T17:30:53.548564+0000","last_peered":"2024-09-17T17:30:53.548564+0000","last_clean":"2024-09-17T17:30:53.548564+0000","last_became_active":"2024-09-17T17:30:26.345748+0000","last_became_peered":"2024-09-17T17:30:26.345748+0000","last_unstale":"2024-09-17T17:30:53.548564+0000","last_undegraded":"2024-09-17T17:30:53.548564+0000","last_fullsized":"2024-09-17T17:30:53.548564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":5,"last_scrub":"65'44","last_scrub_stamp":"2024-09-17T17:30:27.911326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.911326+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.721701+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00059301299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.18","version":"100'31","reported_seq":423,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.549247+0000","last_change":"2024-09-17T17:30:51.551323+0000","last_active":"2024-09-17T17:30:53.549247+0000","last_peered":"2024-09-17T17:30:53.549247+0000","last_clean":"2024-09-17T17:30:53.549247+0000","last_became_active":"2024-09-17T17:30:27.354929+0000","last_became_peered":"2024-09-17T17:30:27.354929+0000","last_unstale":"2024-09-17T17:30:53.549247+0000","last_undegraded":"2024-09-17T17:30:53.549247+0000","last_fullsized":"2024-09-17T17:30:53.549247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":5,"last_scrub":"76'29","last_scrub_stamp":"2024-09-17T17:30:28.377344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.377344+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:36.001747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0050076549999999997,"stat_sum":{"num_bytes":5963776,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":625,"num_write":1,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.17","version":"40'6","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548160+0000","last_change":"2024-09-17T17:30:51.542436+0000","last_active":"2024-09-17T17:30:53.548160+0000","last_peered":"2024-09-17T17:30:53.548160+0000","last_clean":"2024-09-17T17:30:53.548160+0000","last_became_active":"2024-09-17T17:30:19.460414+0000","last_became_peered":"2024-09-17T17:30:19.460414+0000","last_unstale":"2024-09-17T17:30:53.548160+0000","last_undegraded":"2024-09-17T17:30:53.548160+0000","last_fullsized":"2024-09-17T17:30:53.548160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:25.845481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.845481+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:31.818628+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013824040000000001,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.16","version":"100'181","reported_seq":606,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.546058+0000","last_change":"2024-09-17T17:30:51.569089+0000","last_active":"2024-09-17T17:30:53.546058+0000","last_peered":"2024-09-17T17:30:53.546058+0000","last_clean":"2024-09-17T17:30:53.546058+0000","last_became_active":"2024-09-17T17:30:47.363797+0000","last_became_peered":"2024-09-17T17:30:47.363797+0000","last_unstale":"2024-09-17T17:30:53.546058+0000","last_undegraded":"2024-09-17T17:30:53.546058+0000","last_fullsized":"2024-09-17T17:30:53.546058+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'146","last_scrub_stamp":"2024-09-17T17:30:36.055244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.055244+0000","objects_scrubbed":7,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:05.906343+0000","scrub_duration":133,"objects_trimmed":0,"snaptrim_duration":0.028782753000000001,"stat_sum":{"num_bytes":13795328,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":4,"num_read_kb":3,"num_write":13,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.15","version":"100'41","reported_seq":2,"reported_epoch":102,"state":"peering","last_fresh":"2024-09-17T17:30:51.563738+0000","last_change":"2024-09-17T17:30:53.567518+0000","last_active":"2024-09-17T17:30:53.567518+0000","last_peered":"2024-09-17T17:30:51.563738+0000","last_clean":"2024-09-17T17:30:51.563738+0000","last_became_active":"2024-09-17T17:30:46.350418+0000","last_became_peered":"2024-09-17T17:30:46.350418+0000","last_unstale":"2024-09-17T17:30:51.563738+0000","last_undegraded":"2024-09-17T17:30:51.563738+0000","last_fullsized":"2024-09-17T17:30:51.563738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'20","last_scrub_stamp":"2024-09-17T17:30:27.352233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.352233+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.020157329000000002,"stat_sum":{"num_bytes":2080768,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":11,"num_read_kb":1251,"num_write":4,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1736704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0,3,10,4,5,12],"acting":[2147483647,0,3,10,4,5,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":0,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.14","version":"62'42","reported_seq":15,"reported_epoch":102,"state":"peering","last_fresh":"2024-09-17T17:30:53.546866+0000","last_change":"2024-09-17T17:30:53.546866+0000","last_active":"2024-09-17T17:30:53.546866+0000","last_peered":"2024-09-17T17:30:52.555592+0000","last_clean":"2024-09-17T17:30:49.349995+0000","last_became_active":"2024-09-17T17:30:52.554701+0000","last_became_peered":"2024-09-17T17:30:52.554701+0000","last_unstale":"2024-09-17T17:30:53.546866+0000","last_undegraded":"2024-09-17T17:30:53.546866+0000","last_fullsized":"2024-09-17T17:30:53.546866+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"62'42","last_scrub_stamp":"2024-09-17T17:30:26.686769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.686769+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343159999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,15,10,7,2,13],"acting":[3,14,15,10,7,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.550879+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:53.550879+0000","last_peered":"2024-09-17T17:30:53.550879+0000","last_clean":"2024-09-17T17:30:53.550879+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:53.550879+0000","last_undegraded":"2024-09-17T17:30:53.550879+0000","last_fullsized":"2024-09-17T17:30:53.550879+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.115044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"100'151","reported_seq":550,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.546110+0000","last_change":"2024-09-17T17:30:51.570498+0000","last_active":"2024-09-17T17:30:53.546110+0000","last_peered":"2024-09-17T17:30:53.546110+0000","last_clean":"2024-09-17T17:30:53.546110+0000","last_became_active":"2024-09-17T17:30:47.362331+0000","last_became_peered":"2024-09-17T17:30:47.362331+0000","last_unstale":"2024-09-17T17:30:53.546110+0000","last_undegraded":"2024-09-17T17:30:53.546110+0000","last_fullsized":"2024-09-17T17:30:53.546110+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'127","last_scrub_stamp":"2024-09-17T17:30:27.836715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.836715+0000","objects_scrubbed":5,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:52.520132+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.030063706999999999,"stat_sum":{"num_bytes":4784128,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":7,"num_read_kb":669,"num_write":7,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.549084+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:53.549084+0000","last_peered":"2024-09-17T17:30:53.549084+0000","last_clean":"2024-09-17T17:30:53.549084+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:53.549084+0000","last_undegraded":"2024-09-17T17:30:53.549084+0000","last_fullsized":"2024-09-17T17:30:53.549084+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:31.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548283+0000","last_change":"2024-09-17T17:30:51.544193+0000","last_active":"2024-09-17T17:30:53.548283+0000","last_peered":"2024-09-17T17:30:53.548283+0000","last_clean":"2024-09-17T17:30:53.548283+0000","last_became_active":"2024-09-17T17:30:19.457663+0000","last_became_peered":"2024-09-17T17:30:19.457663+0000","last_unstale":"2024-09-17T17:30:53.548283+0000","last_undegraded":"2024-09-17T17:30:53.548283+0000","last_fullsized":"2024-09-17T17:30:53.548283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:24.345150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.345150+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:40.665076+0000","scrub_duration":541,"objects_trimmed":0,"snaptrim_duration":0.0024415970000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T17:30:54.549712+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:54.549712+0000","last_peered":"2024-09-17T17:30:54.549712+0000","last_clean":"2024-09-17T17:30:54.549712+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:54.549712+0000","last_undegraded":"2024-09-17T17:30:54.549712+0000","last_fullsized":"2024-09-17T17:30:54.549712+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:58.071824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'70","reported_seq":479,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T17:30:51.551508+0000","last_change":"2024-09-17T17:30:51.552576+0000","last_active":"2024-09-17T17:30:51.551508+0000","last_peered":"2024-09-17T17:30:51.551508+0000","last_clean":"2024-09-17T17:30:51.551508+0000","last_became_active":"2024-09-17T17:30:46.352101+0000","last_became_peered":"2024-09-17T17:30:46.352101+0000","last_unstale":"2024-09-17T17:30:51.551508+0000","last_undegraded":"2024-09-17T17:30:51.551508+0000","last_fullsized":"2024-09-17T17:30:51.551508+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'48","last_scrub_stamp":"2024-09-17T17:30:27.676614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.676614+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.051338+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.015274865,"stat_sum":{"num_bytes":6848512,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13172736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,15,9,2,6,10,13],"acting":[1,15,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T17:30:51.537594+0000","last_change":"2024-09-17T17:30:47.507145+0000","last_active":"2024-09-17T17:30:51.537594+0000","last_peered":"2024-09-17T17:30:51.537594+0000","last_clean":"2024-09-17T17:30:51.537594+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:30:51.537594+0000","last_undegraded":"2024-09-17T17:30:51.537594+0000","last_fullsized":"2024-09-17T17:30:51.537594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.507074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.507074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.099148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'27","reported_seq":5,"reported_epoch":102,"state":"peering","last_fresh":"2024-09-17T17:30:52.581369+0000","last_change":"2024-09-17T17:30:53.563056+0000","last_active":"2024-09-17T17:30:53.563056+0000","last_peered":"2024-09-17T17:30:52.581369+0000","last_clean":"2024-09-17T17:30:21.347139+0000","last_became_active":"2024-09-17T17:30:52.554994+0000","last_became_peered":"2024-09-17T17:30:52.554994+0000","last_unstale":"2024-09-17T17:30:52.581369+0000","last_undegraded":"2024-09-17T17:30:52.581369+0000","last_fullsized":"2024-09-17T17:30:52.542849+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'27","last_scrub_stamp":"2024-09-17T17:30:28.947105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.947105+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042664590000000002,"stat_sum":{"num_bytes":2981888,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1685,"num_write":22,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,5,2,7,14,3],"acting":[10,0,5,2,7,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"7"}]},{"pgid":"3.3","version":"100'91","reported_seq":491,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.545562+0000","last_change":"2024-09-17T17:30:51.559448+0000","last_active":"2024-09-17T17:30:53.545562+0000","last_peered":"2024-09-17T17:30:53.545562+0000","last_clean":"2024-09-17T17:30:53.545562+0000","last_became_active":"2024-09-17T17:30:26.345813+0000","last_became_peered":"2024-09-17T17:30:26.345813+0000","last_unstale":"2024-09-17T17:30:53.545562+0000","last_undegraded":"2024-09-17T17:30:53.545562+0000","last_fullsized":"2024-09-17T17:30:53.545562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'58","last_scrub_stamp":"2024-09-17T17:30:28.942498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.942498+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:55.401727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.018839663,"stat_sum":{"num_bytes":7880704,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":2592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.2","version":"28'2","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.545950+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:53.545950+0000","last_peered":"2024-09-17T17:30:53.545950+0000","last_clean":"2024-09-17T17:30:53.545950+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:53.545950+0000","last_undegraded":"2024-09-17T17:30:53.545950+0000","last_fullsized":"2024-09-17T17:30:53.545950+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:53.706511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"100'96","reported_seq":486,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548397+0000","last_change":"2024-09-17T17:30:51.555622+0000","last_active":"2024-09-17T17:30:53.548397+0000","last_peered":"2024-09-17T17:30:53.548397+0000","last_clean":"2024-09-17T17:30:53.548397+0000","last_became_active":"2024-09-17T17:30:27.343750+0000","last_became_peered":"2024-09-17T17:30:27.343750+0000","last_unstale":"2024-09-17T17:30:53.548397+0000","last_undegraded":"2024-09-17T17:30:53.548397+0000","last_fullsized":"2024-09-17T17:30:53.548397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'62","last_scrub_stamp":"2024-09-17T17:30:28.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.961363+0000","objects_scrubbed":3,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:38.115499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.018887095999999999,"stat_sum":{"num_bytes":4014080,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":10,"num_read_kb":6919,"num_write":5,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":175,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.550734+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:30:53.550734+0000","last_peered":"2024-09-17T17:30:53.550734+0000","last_clean":"2024-09-17T17:30:53.550734+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:53.550734+0000","last_undegraded":"2024-09-17T17:30:53.550734+0000","last_fullsized":"2024-09-17T17:30:53.550734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"100'55","reported_seq":442,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548076+0000","last_change":"2024-09-17T17:30:51.555660+0000","last_active":"2024-09-17T17:30:53.548076+0000","last_peered":"2024-09-17T17:30:53.548076+0000","last_clean":"2024-09-17T17:30:53.548076+0000","last_became_active":"2024-09-17T17:30:26.354825+0000","last_became_peered":"2024-09-17T17:30:26.354825+0000","last_unstale":"2024-09-17T17:30:53.548076+0000","last_undegraded":"2024-09-17T17:30:53.548076+0000","last_fullsized":"2024-09-17T17:30:53.548076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:28.852921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.852921+0000","objects_scrubbed":2,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:56.003445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.014426430000000001,"stat_sum":{"num_bytes":3211264,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":8117,"num_write":6,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":175,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.545843+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:30:53.545843+0000","last_peered":"2024-09-17T17:30:53.545843+0000","last_clean":"2024-09-17T17:30:53.545843+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:53.545843+0000","last_undegraded":"2024-09-17T17:30:53.545843+0000","last_fullsized":"2024-09-17T17:30:53.545843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"63'42","reported_seq":400,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.550652+0000","last_change":"2024-09-17T17:30:51.549246+0000","last_active":"2024-09-17T17:30:53.550652+0000","last_peered":"2024-09-17T17:30:53.550652+0000","last_clean":"2024-09-17T17:30:53.550652+0000","last_became_active":"2024-09-17T17:30:46.353966+0000","last_became_peered":"2024-09-17T17:30:46.353966+0000","last_unstale":"2024-09-17T17:30:53.550652+0000","last_undegraded":"2024-09-17T17:30:53.550652+0000","last_fullsized":"2024-09-17T17:30:53.550652+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'42","last_scrub_stamp":"2024-09-17T17:30:25.590903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.590903+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:54.863995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966410000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0,3,4,15,10,11],"acting":[14,0,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"2.3","version":"26'1","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.550707+0000","last_change":"2024-09-17T17:30:38.649965+0000","last_active":"2024-09-17T17:30:53.550707+0000","last_peered":"2024-09-17T17:30:53.550707+0000","last_clean":"2024-09-17T17:30:53.550707+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:53.550707+0000","last_undegraded":"2024-09-17T17:30:53.550707+0000","last_fullsized":"2024-09-17T17:30:53.550707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:30:38.649873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.649873+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.468836+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":310,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.549498+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:30:53.549498+0000","last_peered":"2024-09-17T17:30:53.549498+0000","last_clean":"2024-09-17T17:30:53.549498+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:53.549498+0000","last_undegraded":"2024-09-17T17:30:53.549498+0000","last_fullsized":"2024-09-17T17:30:53.549498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"75'29","reported_seq":418,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.549401+0000","last_change":"2024-09-17T17:30:51.554183+0000","last_active":"2024-09-17T17:30:53.549401+0000","last_peered":"2024-09-17T17:30:53.549401+0000","last_clean":"2024-09-17T17:30:53.549401+0000","last_became_active":"2024-09-17T17:30:26.348354+0000","last_became_peered":"2024-09-17T17:30:26.348354+0000","last_unstale":"2024-09-17T17:30:53.549401+0000","last_undegraded":"2024-09-17T17:30:53.549401+0000","last_fullsized":"2024-09-17T17:30:53.549401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'29","last_scrub_stamp":"2024-09-17T17:30:27.407808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.407808+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.453798+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0079920400000000006,"stat_sum":{"num_bytes":344064,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.9","version":"100'75","reported_seq":485,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548831+0000","last_change":"2024-09-17T17:30:51.549010+0000","last_active":"2024-09-17T17:30:53.548831+0000","last_peered":"2024-09-17T17:30:53.548831+0000","last_clean":"2024-09-17T17:30:53.548831+0000","last_became_active":"2024-09-17T17:30:28.394697+0000","last_became_peered":"2024-09-17T17:30:28.394697+0000","last_unstale":"2024-09-17T17:30:53.548831+0000","last_undegraded":"2024-09-17T17:30:53.548831+0000","last_fullsized":"2024-09-17T17:30:53.548831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'61","last_scrub_stamp":"2024-09-17T17:30:36.024221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.024221+0000","objects_scrubbed":4,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:44.846851+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.012117802,"stat_sum":{"num_bytes":9404416,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":5,"num_read_kb":12170,"num_write":2,"num_write_kb":2336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.a","version":"100'140","reported_seq":485,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.545915+0000","last_change":"2024-09-17T17:30:51.809656+0000","last_active":"2024-09-17T17:30:53.545915+0000","last_peered":"2024-09-17T17:30:53.545915+0000","last_clean":"2024-09-17T17:30:53.545915+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:53.545915+0000","last_undegraded":"2024-09-17T17:30:53.545915+0000","last_fullsized":"2024-09-17T17:30:53.545915+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'140","last_scrub_stamp":"2024-09-17T17:30:51.809533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.809533+0000","objects_scrubbed":8,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:54.076947+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.017590710999999998,"stat_sum":{"num_bytes":14680064,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":52,"num_read_kb":15272,"num_write":108,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.b","version":"96'22","reported_seq":383,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548444+0000","last_change":"2024-09-17T17:30:51.542828+0000","last_active":"2024-09-17T17:30:53.548444+0000","last_peered":"2024-09-17T17:30:53.548444+0000","last_clean":"2024-09-17T17:30:53.548444+0000","last_became_active":"2024-09-17T17:30:47.358299+0000","last_became_peered":"2024-09-17T17:30:47.358299+0000","last_unstale":"2024-09-17T17:30:53.548444+0000","last_undegraded":"2024-09-17T17:30:53.548444+0000","last_fullsized":"2024-09-17T17:30:53.548444+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:12.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001376436,"stat_sum":{"num_bytes":2310144,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":6719,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4538368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,10,6,8,7,14],"acting":[13,3,10,6,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.c","version":"100'174","reported_seq":501,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548754+0000","last_change":"2024-09-17T17:30:51.567084+0000","last_active":"2024-09-17T17:30:53.548754+0000","last_peered":"2024-09-17T17:30:53.548754+0000","last_clean":"2024-09-17T17:30:53.548754+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:53.548754+0000","last_undegraded":"2024-09-17T17:30:53.548754+0000","last_fullsized":"2024-09-17T17:30:53.548754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:01.541591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026776215999999999,"stat_sum":{"num_bytes":17776640,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":50,"num_read_kb":11988,"num_write":102,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.d","version":"100'110","reported_seq":427,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T17:30:52.546027+0000","last_change":"2024-09-17T17:30:51.569527+0000","last_active":"2024-09-17T17:30:52.546027+0000","last_peered":"2024-09-17T17:30:52.546027+0000","last_clean":"2024-09-17T17:30:52.546027+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:30:52.546027+0000","last_undegraded":"2024-09-17T17:30:52.546027+0000","last_fullsized":"2024-09-17T17:30:52.546027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:11.750198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.025471486000000002,"stat_sum":{"num_bytes":9322496,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":19,"num_read_kb":9229,"num_write":58,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.e","version":"100'66","reported_seq":395,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T17:30:54.549574+0000","last_change":"2024-09-17T17:30:52.137405+0000","last_active":"2024-09-17T17:30:54.549574+0000","last_peered":"2024-09-17T17:30:54.549574+0000","last_clean":"2024-09-17T17:30:54.549574+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:54.549574+0000","last_undegraded":"2024-09-17T17:30:54.549574+0000","last_fullsized":"2024-09-17T17:30:54.549574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:01.710224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.590164252,"stat_sum":{"num_bytes":6520832,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":18,"num_read_kb":3129,"num_write":52,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.f","version":"100'102","reported_seq":522,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548946+0000","last_change":"2024-09-17T17:30:51.566879+0000","last_active":"2024-09-17T17:30:53.548946+0000","last_peered":"2024-09-17T17:30:53.548946+0000","last_clean":"2024-09-17T17:30:53.548946+0000","last_became_active":"2024-09-17T17:30:47.356042+0000","last_became_peered":"2024-09-17T17:30:47.356042+0000","last_unstale":"2024-09-17T17:30:53.548946+0000","last_undegraded":"2024-09-17T17:30:53.548946+0000","last_fullsized":"2024-09-17T17:30:53.548946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:13.266118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.025965390000000001,"stat_sum":{"num_bytes":15581184,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":74,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":41320448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,4,14,2,3,0],"acting":[11,8,4,14,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.10","version":"77'53","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.549170+0000","last_change":"2024-09-17T17:30:51.556879+0000","last_active":"2024-09-17T17:30:53.549170+0000","last_peered":"2024-09-17T17:30:53.549170+0000","last_clean":"2024-09-17T17:30:53.549170+0000","last_became_active":"2024-09-17T17:30:47.354125+0000","last_became_peered":"2024-09-17T17:30:47.354125+0000","last_unstale":"2024-09-17T17:30:53.549170+0000","last_undegraded":"2024-09-17T17:30:53.549170+0000","last_fullsized":"2024-09-17T17:30:53.549170+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'52","last_scrub_stamp":"2024-09-17T17:30:26.907313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.907313+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.061590+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.011082936,"stat_sum":{"num_bytes":2342912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7,14,5,9,11],"acting":[6,0,7,14,5,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.11","version":"60'25","reported_seq":123,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T17:30:54.543805+0000","last_change":"2024-09-17T17:30:51.540542+0000","last_active":"2024-09-17T17:30:54.543805+0000","last_peered":"2024-09-17T17:30:54.543805+0000","last_clean":"2024-09-17T17:30:54.543805+0000","last_became_active":"2024-09-17T17:30:36.353414+0000","last_became_peered":"2024-09-17T17:30:36.353414+0000","last_unstale":"2024-09-17T17:30:54.543805+0000","last_undegraded":"2024-09-17T17:30:54.543805+0000","last_fullsized":"2024-09-17T17:30:54.543805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:37.309789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:37.309789+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:40.299384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079811599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,10,5,14,2],"acting":[4,11,15,10,5,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.12","version":"100'95","reported_seq":502,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.550515+0000","last_change":"2024-09-17T17:30:51.564660+0000","last_active":"2024-09-17T17:30:53.550515+0000","last_peered":"2024-09-17T17:30:53.550515+0000","last_clean":"2024-09-17T17:30:53.550515+0000","last_became_active":"2024-09-17T17:30:39.350802+0000","last_became_peered":"2024-09-17T17:30:39.350802+0000","last_unstale":"2024-09-17T17:30:53.550515+0000","last_undegraded":"2024-09-17T17:30:53.550515+0000","last_fullsized":"2024-09-17T17:30:53.550515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'48","last_scrub_stamp":"2024-09-17T17:30:24.093396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.093396+0000","objects_scrubbed":3,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:35.435680+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0.018394490999999999,"stat_sum":{"num_bytes":3489792,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":3,"num_read_kb":1,"num_write":11,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6160384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,7,5,4,11,3,15],"acting":[14,7,5,4,11,3,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]},{"pgid":"3.13","version":"100'68","reported_seq":492,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T17:30:53.548929+0000","last_change":"2024-09-17T17:30:51.552659+0000","last_active":"2024-09-17T17:30:53.548929+0000","last_peered":"2024-09-17T17:30:53.548929+0000","last_clean":"2024-09-17T17:30:53.548929+0000","last_became_active":"2024-09-17T17:30:46.350583+0000","last_became_peered":"2024-09-17T17:30:46.350583+0000","last_unstale":"2024-09-17T17:30:53.548929+0000","last_undegraded":"2024-09-17T17:30:53.548929+0000","last_fullsized":"2024-09-17T17:30:53.548929+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'52","last_scrub_stamp":"2024-09-17T17:30:27.908152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.908152+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:41.948407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.012392781,"stat_sum":{"num_bytes":4063232,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7634944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,6,5,4,13],"acting":[11,7,2,6,5,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"2"},{"start":"19","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":137986048,"num_objects":115,"num_object_clones":65,"num_object_copies":805,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":17,"num_read":234,"num_read_kb":78072,"num_write":508,"num_write_kb":95968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":81362944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":254951424,"data_stored":372764672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1842,"ondisk_log_size":1842,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247828,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47508,"kb_used_data":12152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324332,"statfs":{"total":96636764160,"available":96588115968,"internally_reserved":0,"allocated":12443648,"data_stored":20748637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247828,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51148,"kb_used_data":19952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":20430848,"data_stored":29864340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247829,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46912,"kb_used_data":15720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324928,"statfs":{"total":96636764160,"available":96588726272,"internally_reserved":0,"allocated":16097280,"data_stored":15944029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313236,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51424,"kb_used_data":16068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94320416,"statfs":{"total":96636764160,"available":96584105984,"internally_reserved":0,"allocated":16453632,"data_stored":23678834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45952,"kb_used_data":14756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325888,"statfs":{"total":96636764160,"available":96589709312,"internally_reserved":0,"allocated":15110144,"data_stored":23904660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313237,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48612,"kb_used_data":17424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323228,"statfs":{"total":96636764160,"available":96586985472,"internally_reserved":0,"allocated":17842176,"data_stored":22804688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280533,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52328,"kb_used_data":21132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319512,"statfs":{"total":96636764160,"available":96583180288,"internally_reserved":0,"allocated":21639168,"data_stored":26338228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46908,"kb_used_data":15720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324932,"statfs":{"total":96636764160,"available":96588730368,"internally_reserved":0,"allocated":16097280,"data_stored":22933908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50872,"kb_used_data":15520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94320968,"statfs":{"total":96636764160,"available":96584671232,"internally_reserved":0,"allocated":15892480,"data_stored":20435348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280533,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48952,"kb_used_data":13600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322888,"statfs":{"total":96636764160,"available":96586637312,"internally_reserved":0,"allocated":13926400,"data_stored":22606247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280532,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44132,"kb_used_data":12944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327708,"statfs":{"total":96636764160,"available":96591572992,"internally_reserved":0,"allocated":13254656,"data_stored":19304852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280532,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":16876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":17281024,"data_stored":22434196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280532,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49392,"kb_used_data":22364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322448,"statfs":{"total":96636764160,"available":96586186752,"internally_reserved":0,"allocated":22900736,"data_stored":34795924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54440,"kb_used_data":23252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317400,"statfs":{"total":96636764160,"available":96581017600,"internally_reserved":0,"allocated":23810048,"data_stored":28696999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247829,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70452,"kb_used_data":17048,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94301388,"statfs":{"total":96636764160,"available":96564621312,"internally_reserved":0,"allocated":17457152,"data_stored":27773277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247828,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50224,"kb_used_data":19028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321616,"statfs":{"total":96636764160,"available":96585334784,"internally_reserved":0,"allocated":19484672,"data_stored":23634868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19611648,"data_stored":24932352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14991360,"data_stored":23920640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21405696,"data_stored":33918976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22310912,"data_stored":27820032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15945728,"data_stored":26886144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14397440,"data_stored":19558400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17457152,"data_stored":22228992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16715776,"data_stored":23322624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14057472,"data_stored":14528512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":12427264,"data_stored":21729280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":11759616,"data_stored":18427904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":18935808,"data_stored":28987392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":10932224,"data_stored":19861504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":14602240,"data_stored":22056960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":13615104,"data_stored":23027712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":15785984,"data_stored":21557248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:30:56.401 INFO:tasks.thrashosds.thrasher:Setting 3.f to [13, 14, 5, 10, 0, 9, 6] 2024-09-17T17:30:56.401 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.f', '13', '14', '5', '10', '0', '9', '6'] 2024-09-17T17:30:56.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.f 13 14 5 10 0 9 6 2024-09-17T17:30:56.441 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:56.439+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:56.495 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.496 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:30:56.542 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:56.540+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:56.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:30:56.670 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:56.668+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:56.743 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:56.741+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:56.753 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:56.792 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:56.810 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:56.810 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:56.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:30:56.820 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:56.842 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:56.844 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:56.842+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:56.863 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:56.863 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:56.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:56.886 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:56.907 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:56.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:56.930 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:56.930 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:56.930 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:56.930 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:56.930 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:56.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:30:56.945 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:30:56.943+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:30:56.949 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:56.950 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:56.973 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:56.973 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:56.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:57.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:57.038 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:57.045 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:57.043+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:57.059 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:57.059 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:57.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:57.059 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:57.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.059 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=94, reqid=client.4566.0:406, at_version=51'30, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.713502544999997, 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003277556, 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343212+0000", 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343225+0000", 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414999999999999e-05 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343249+0000", 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3983999999999999e-05 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343254+0000", 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5249999999999996e-06 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343307+0000", 2024-09-17T17:30:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2167999999999999e-05 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343315+0000", 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8580000000000006e-06 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346453+0000", 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003137908 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346488+0000", 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5021999999999998e-05 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 56/35 ECSubWrite(tid=57, reqid=osd.10.0:57, at_version=56'41, trim_to=0'0, roll_forward_to=53'39))", 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.721449970000002, 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036979209999999998, 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:30:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335264+0000", 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335298+0000", 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3837000000000001e-05 2024-09-17T17:30:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335654+0000", 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035630699999999999 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335665+0000", 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1620999999999999e-05 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335751+0000", 2024-09-17T17:30:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5692999999999994e-05 2024-09-17T17:30:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335763+0000", 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140000000000001e-05 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338926+0000", 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031629399999999999 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338961+0000", 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5129999999999997e-05 2024-09-17T17:30:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.718644255999997, 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003185701, 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:30:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338094+0000", 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4428000000000001e-05 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011644700000000001 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338219+0000", 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9020000000000005e-06 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338416+0000", 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196558 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338424+0000", 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.585e-06 2024-09-17T17:30:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341209+0000", 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027847050000000002 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341255+0000", 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.718603025999997, 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004124801, 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9600000000000001e-07 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338129+0000", 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8003e-05 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338243+0000", 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114367 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338249+0000", 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3160000000000002e-06 2024-09-17T17:30:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338788+0000", 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053876399999999995 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338797+0000", 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-06 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342196+0000", 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033993830000000002 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342235+0000", 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8921999999999998e-05 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.718504092000003, 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040538249999999996, 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.99e-07 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338233+0000", 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3623999999999999e-05 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338275+0000", 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2249000000000002e-05 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338284+0000", 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2609999999999993e-06 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338436+0000", 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152032 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338448+0000", 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2469999999999999e-05 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342230+0000", 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037816899999999999 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342263+0000", 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3300000000000003e-05 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.718443481000001, 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029941270000000001, 2024-09-17T17:30:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338289+0000", 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8340000000000001e-05 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338322+0000", 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3419e-05 2024-09-17T17:30:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338332+0000", 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0086e-05 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338873+0000", 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054120000000000004 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338882+0000", 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1329999999999999e-06 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341233+0000", 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023510699999999998 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1365999999999998e-05 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.684338648999997, 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285746, 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372399+0000", 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4416000000000001e-05 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372479+0000", 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9556999999999997e-05 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372487+0000", 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7589999999999997e-06 2024-09-17T17:30:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372561+0000", 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3807000000000007e-05 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372571+0000", 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0532e-05 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375622+0000", 2024-09-17T17:30:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030507860000000002 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375661+0000", 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8634000000000003e-05 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.683608882999998, 2024-09-17T17:30:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004179131, 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373125+0000", 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9626000000000001e-05 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373159+0000", 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4190999999999998e-05 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373168+0000", 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6999999999999997e-06 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373237+0000", 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9306000000000004e-05 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373246+0000", 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377245+0000", 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039987390000000003 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377284+0000", 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8423999999999997e-05 2024-09-17T17:30:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.683262765000002, 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048101309999999996, 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1800000000000005e-07 2024-09-17T17:30:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373476+0000", 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4712999999999999e-05 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373535+0000", 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8536999999999999e-05 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373545+0000", 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0142e-05 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374163+0000", 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061771899999999997 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374172+0000", 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.081e-06 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378233+0000", 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040611509999999998 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378261+0000", 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000002e-05 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.683212679999997, 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038195389999999998, 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373518+0000", 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373595+0000", 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6206999999999997e-05 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373601+0000", 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6039999999999998e-06 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373659+0000", 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7442000000000003e-05 2024-09-17T17:30:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373668+0000", 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2040000000000003e-06 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377300+0000", 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00363182 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377320+0000", 2024-09-17T17:30:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0598000000000002e-05 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.680611155000001, 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034371530000000001, 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376103+0000", 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376128+0000", 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5109e-05 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376199+0000", 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1783e-05 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376209+0000", 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1710000000000008e-06 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376272+0000", 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749000000000002e-05 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376281+0000", 2024-09-17T17:30:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379498+0000", 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003216605 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379539+0000", 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1587e-05 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 68/67 ECSubWrite(tid=75, reqid=client.4566.0:644, at_version=68'52, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.356766575999998, 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52365443700000003, 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699948+0000", 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999998e-07 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699964+0000", 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6784000000000001e-05 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222607+0000", 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52264301099999999 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222621+0000", 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3923e-05 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222745+0000", 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012366800000000001 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222753+0000", 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6490000000000008e-06 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223566+0000", 2024-09-17T17:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081285500000000002 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223601+0000", 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5744000000000002e-05 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/74 ECSubWrite(tid=163, reqid=osd.6.0:163, at_version=75'29, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.705651604, 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.34608951199999999, 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:30:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351063+0000", 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9999999999999997e-07 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351085+0000", 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2129000000000001e-05 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696028+0000", 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.344942945 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696043+0000", 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5353e-05 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696175+0000", 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131996 2024-09-17T17:30:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696189+0000", 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3573e-05 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697107+0000", 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091841899999999996 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697151+0000", 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4397000000000002e-05 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.13s4 80/74 ECSubWrite(tid=111, reqid=osd.10.0:111, at_version=80'62, trim_to=0'0, roll_forward_to=79'60))", 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.709293031000001, 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003050119, 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347421+0000", 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.03e-07 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347462+0000", 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0862999999999998e-05 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347534+0000", 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2643000000000002e-05 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347550+0000", 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5633000000000001e-05 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347706+0000", 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015574400000000001 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347737+0000", 2024-09-17T17:30:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1164000000000002e-05 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350445+0000", 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027080540000000001 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350470+0000", 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5715e-05 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 92/76 ECSubWrite(tid=213, reqid=osd.12.0:213, at_version=92'67, trim_to=0'0, roll_forward_to=91'65))", 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700554744, 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049639100000000002, 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356183+0000", 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4491999999999999e-05 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356228+0000", 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5090999999999999e-05 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356237+0000", 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8950000000000004e-06 2024-09-17T17:30:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356968+0000", 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073084099999999998 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356978+0000", 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4069999999999999e-06 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.361091+0000", 2024-09-17T17:30:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0041133680000000001 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.361123+0000", 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1605e-05 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 92/74 ECSubWrite(tid=128, reqid=osd.10.0:128, at_version=92'78, trim_to=0'0, roll_forward_to=85'76))", 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:30:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700504652999999, 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031162680000000002, 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:30:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0900000000000001e-07 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356229+0000", 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0058999999999999e-05 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356293+0000", 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4162e-05 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356301+0000", 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.401e-06 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356479+0000", 2024-09-17T17:30:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000178297 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356488+0000", 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2259999999999994e-06 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.359287+0000", 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027985950000000001 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.359325+0000", 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8318999999999997e-05 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.12s3 92/87 ECSubWrite(tid=86, reqid=osd.14.0:86, at_version=92'78, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.699996783, 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039887689999999996, 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6300000000000005e-07 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356738+0000", 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1166e-05 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356812+0000", 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3659999999999996e-05 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356820+0000", 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9319999999999996e-06 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.357010+0000", 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019038799999999999 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.357020+0000", 2024-09-17T17:30:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7929999999999992e-06 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.360667+0000", 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003646935 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.360705+0000", 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8331999999999998e-05 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 96/74 ECSubWrite(tid=130, reqid=client.4566.0:925, at_version=96'80, trim_to=0'0, roll_forward_to=92'78))", 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.6761178050000005, 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037922670000000002, 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7100000000000001e-07 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.382960+0000", 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002364395 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.382989+0000", 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8736000000000001e-05 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.382998+0000", 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7600000000000008e-06 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.383057+0000", 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9126000000000003e-05 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.383065+0000", 2024-09-17T17:30:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0479999999999999e-06 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.384360+0000", 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012951690000000001 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.384388+0000", 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7861999999999999e-05 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.6783778389999995, 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029972480000000001, 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:30:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378747+0000", 2024-09-17T17:30:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041115500000000002 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378799+0000", 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1851000000000002e-05 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378806+0000", 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.435e-06 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378889+0000", 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2730000000000002e-05 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378904+0000", 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4552000000000001e-05 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.381296+0000", 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023924250000000001 2024-09-17T17:30:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.381333+0000", 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6878000000000002e-05 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.18s3 100/75 ECSubWrite(tid=176, reqid=client.4566.0:964, at_version=100'31, trim_to=0'0, roll_forward_to=76'29))", 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:51.787785+0000", 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2689281189999999, 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057844150000000002, 2024-09-17T17:30:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787785+0000", 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787785+0000", 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787786+0000", 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.16e-07 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791036+0000", 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003250368 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791078+0000", 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1631000000000002e-05 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791087+0000", 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1970000000000002e-06 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791154+0000", 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6576999999999996e-05 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791164+0000", 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0282000000000001e-05 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.793526+0000", 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023622949999999999 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.793570+0000", 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3248999999999999e-05 2024-09-17T17:30:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.155 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:57.155 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:30:57.063+0000 7fd0daffd640 0 -- 172.21.15.7:0/4126260238 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fd0dc2216a0 msgr2=0x7fd0dc223a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:30:57.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:30:57.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:57.158 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:30:57.198 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:57.196+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:57.260 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:57.258+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:57.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-17T17:30:57.360 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:57.358+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:57.420 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:57.421 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:57.421 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:57.421 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:57.421 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:57.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-17T17:30:57.462 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:57.460+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:57.562 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:30:57.561+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:30:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.f pg_upmap mapping to [13,14,5,10,0,9,6] 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.891105594999999, 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029476239999999998, 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684657+0000", 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.684688+0000", 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1089e-05 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685268+0000", 2024-09-17T17:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058046599999999997 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685278+0000", 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6010000000000001e-06 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685351+0000", 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3164000000000006e-05 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685362+0000", 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0757999999999999e-05 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687544+0000", 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021819439999999999 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.687604+0000", 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340999999999997e-05 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:199 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~540672] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.877569425999994, 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071389620000000004, 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 199 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698193+0000", 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698195+0000", 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698857+0000", 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066281499999999998 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698878+0000", 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0792e-05 2024-09-17T17:30:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698891+0000", 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3040000000000001e-05 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.698980+0000", 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8523999999999995e-05 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.699087+0000", 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010695 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700340+0000", 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00125289 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703104+0000", 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00276414 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705274+0000", 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021704509999999999 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705332+0000", 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7463000000000002e-05 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:200 3.es0 3:71a2b96a:::smithi10238019-23:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.875343567000002, 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051229079999999998, 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 200 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700419+0000", 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700420+0000", 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-17T17:30:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700796+0000", 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037557600000000002 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700812+0000", 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5974999999999999e-05 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700822+0000", 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7380000000000006e-06 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701247+0000", 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042503000000000001 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701308+0000", 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1148000000000005e-05 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.701878+0000", 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057030600000000002 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703209+0000", 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001331404 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705465+0000", 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022554329999999998 2024-09-17T17:30:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705541+0000", 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6643000000000005e-05 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:201 3.es0 3:71a2b96a:::smithi10238019-23:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.874919488000003, 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048457270000000002, 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 201 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700843+0000", 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700844+0000", 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4600000000000003e-07 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700862+0000", 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8513e-05 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700881+0000", 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9317000000000002e-05 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700888+0000", 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1340000000000001e-06 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702439+0000", 2024-09-17T17:30:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015509530000000001 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702509+0000", 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0475000000000005e-05 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.702800+0000", 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029064200000000003 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703384+0000", 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058405800000000004 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705636+0000", 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022526780000000001 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.705688+0000", 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2011000000000001e-05 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:202 3.es0 3:71a2b96a:::smithi10238019-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.874825185999995, 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060638539999999996, 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 202 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700937+0000", 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700938+0000", 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700943+0000", 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6470000000000003e-06 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700981+0000", 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8031000000000002e-05 2024-09-17T17:30:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700998+0000", 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6623999999999999e-05 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703475+0000", 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024764850000000001 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.703525+0000", 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0899000000000002e-05 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706147+0000", 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026217829999999999 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.706228+0000", 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0371999999999996e-05 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.707001+0000", 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077321400000000004 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=82, reqid=osd.3.0:82, at_version=57'61, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.204487331999999, 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035036519999999999, 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371275+0000", 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-07 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.371301+0000", 2024-09-17T17:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6461e-05 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372066+0000", 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076444900000000005 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372076+0000", 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372145+0000", 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9586999999999996e-05 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372155+0000", 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0054999999999999e-05 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374734+0000", 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790520000000001 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374778+0000", 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4042000000000003e-05 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.as4 57/32 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=57'63, trim_to=0'0, roll_forward_to=56'59))", 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.202398035000002, 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026238189999999999, 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373364+0000", 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373365+0000", 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0640000000000001e-06 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373399+0000", 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3757000000000002e-05 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373439+0000", 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0028999999999999e-05 2024-09-17T17:30:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373469+0000", 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0297000000000001e-05 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373543+0000", 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4202999999999994e-05 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373574+0000", 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0895999999999997e-05 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375923+0000", 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023489689999999998 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375988+0000", 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4603999999999993e-05 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:581 3.es0 3:7a426564:::smithi10238019-31:head [create,setxattr TahZbq^WXaahVeQfQsXqNXgcd (1963) in=25b,setxattr `Pj`q[kZqXoPYYj_`hNV^bohgce[Ull^lRXTctln (982) in=40b,setxattr ^gfWYcnTRmaWZQkrf`asZ\\XVPcRmRdRckrhUheci (10312) in=40b,setxattr hVm[Q^c]jmYfcedO_eoe[N_fohZ^nghUPgNQYY`h (15648) in=40b,setxattr \\jlYtrYeh_rthhUaqacc]`Uger\\]rPngbjPhbaWV (454) in=40b,setxattr nOgORnqOheegXnS]tSkTQTreXqqt`ojnVPsQbabf (771) in=40b,setxattr QR]c[o[]SToV\\goc\\pjtRRf`WYSngR[eQ`taQgcc (12488) in=40b,setxattr pY\\tNq\\a[n\\fajnSVYNng[\\kcqZ^VhqNmqRXUTlO (1121) in=40b,setxattr [laaiUa\\cY`kdXP\\SNRq[ig]rcanlRgnTtdfZi_W (1403) in=40b,setxattr aaf[PNUURnfSbT^fR_ge[lfbq]keQjYgrbO\\SbTa (1226) in=40b,setxattr SfaWR`YpWpbnn]sR]^eO[Wbi\\Se_TkrcQp[pcomR (4768) in=40b,setxattr pS^PVQeeOqs_S^YVlb^n\\h``m_Q\\prek^_qjcmgR (1121) in=40b,setxattr [aWfmed^jZheSjc^Xmb]TrYaU[[c]jb\\[UdNQngi (593) in=40b,setxattr kPO^emkeWSaQtnWaRVXk[hbN[TP^]ThcqpOfordV (2805) in=40b,setxattr PkhW`mV]l_O[nUSprX^tRPoRakfUNbOiRedpZb^n (421) in=40b,setxattr [`esP (1963) in=5b,setxattr SlhkjUdadks\\hNrmRohr_fl`itUjb_Vjia\\Vdqf` (10033) in=40b,setxattr TUWTXRdZcQc\\lOTbYmrWiirilafTQQrNdPoj_^sZ (3084) in=40b,setxattr ddbfaqshSg (1963) in=10b,setxattr nSgahmseaQO^hNUcamf\\m[SsQam^Uc``at``i`kU (1435) in=40b] snapc 11=[11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.213894830999998, 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047340350000000002, 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 581 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361867+0000", 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.361869+0000", 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.623e-06 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362010+0000", 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014150499999999999 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362027+0000", 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6569e-05 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362040+0000", 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2767e-05 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362112+0000", 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2401e-05 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.362122+0000", 2024-09-17T17:30:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0013000000000001e-05 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.363809+0000", 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016863329999999999 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.363887+0000", 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8323000000000002e-05 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.364760+0000", 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087325800000000002 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.365703+0000", 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094332599999999997 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.366501+0000", 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000797326 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:16.366601+0000", 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100591 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:656 3.es0 3:7a426564:::smithi10238019-31:head [append 0~0] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.228245745999999, 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081233910000000006, 2024-09-17T17:30:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 656 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347516+0000", 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347519+0000", 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6850000000000001e-06 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347531+0000", 2024-09-17T17:30:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2339e-05 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347553+0000", 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2005999999999999e-05 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347564+0000", 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0944999999999999e-05 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347643+0000", 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8885000000000001e-05 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347652+0000", 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350793+0000", 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031410100000000001 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.350899+0000", 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105824 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.351778+0000", 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087886400000000003 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353482+0000", 2024-09-17T17:30:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001703577 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355585+0000", 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021028700000000002 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355640+0000", 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5186999999999999e-05 2024-09-17T17:30:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:657 3.es0 3:7a426564:::smithi10238019-31:head [rollback f] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.228166461000001, 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088160730000000007, 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 657 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347596+0000", 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7800000000000001e-07 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347600+0000", 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.501e-06 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347608+0000", 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5990000000000002e-06 2024-09-17T17:30:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347615+0000", 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347721+0000", 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106421 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:30:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347727+0000", 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0349999999999998e-06 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.352452+0000", 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047242070000000002 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.352548+0000", 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5958000000000005e-05 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353151+0000", 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060365899999999999 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354055+0000", 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090400699999999997 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356330+0000", 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022750029999999998 2024-09-17T17:30:57.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356412+0000", 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1454999999999999e-05 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:726 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~606208] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.224159333000003, 2024-09-17T17:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049411350000000001, 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 726 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351603+0000", 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.351604+0000", 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6700000000000001e-06 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352206+0000", 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060191499999999996 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352220+0000", 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.342e-05 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352227+0000", 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5989999999999996e-06 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352307+0000", 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9568999999999997e-05 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.352377+0000", 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9842e-05 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.353455+0000", 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001077801 2024-09-17T17:30:57.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354825+0000", 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001370509 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356461+0000", 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016354329999999999 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356544+0000", 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3376999999999998e-05 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:727 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~507904] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.221192875, 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027356989999999999, 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 727 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354569+0000", 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.354571+0000", 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.302e-06 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355037+0000", 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046691300000000001 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355070+0000", 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2064000000000003e-05 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355079+0000", 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8260000000000005e-06 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355121+0000", 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1813000000000001e-05 2024-09-17T17:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355196+0000", 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4367000000000001e-05 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355749+0000", 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055320300000000005 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.356631+0000", 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00088253400000000003 2024-09-17T17:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357264+0000", 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063273500000000005 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357305+0000", 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0942000000000002e-05 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:728 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~573440] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.217955633999999, 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028748519999999998, 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 728 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.648 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357807+0000", 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.357808+0000", 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3650000000000001e-06 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358354+0000", 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054612300000000003 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358365+0000", 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1242e-05 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358371+0000", 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5219999999999998e-06 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358404+0000", 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139000000000002e-05 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358466+0000", 2024-09-17T17:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2359000000000004e-05 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.358974+0000", 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050814800000000004 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.359801+0000", 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082624900000000004 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360638+0000", 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083704500000000002 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360681+0000", 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3659999999999999e-05 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:729 3.es0 3:74f6cb56:::smithi10238019-48:head [append 0~475136] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:57.653 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.215252691000003, 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034332189999999999, 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 729 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360509+0000", 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360510+0000", 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0699999999999996e-07 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360948+0000", 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000437978 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360962+0000", 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3478999999999999e-05 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360969+0000", 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4680000000000003e-06 2024-09-17T17:30:57.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361023+0000", 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3381999999999998e-05 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361088+0000", 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5579000000000004e-05 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361618+0000", 2024-09-17T17:30:57.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052954599999999999 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362780+0000", 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001162787 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363897+0000", 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011169929999999999 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363943+0000", 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000001e-05 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:730 3.es0 3:74f6cb56:::smithi10238019-48:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.214775156999998, 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030870279999999999, 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 730 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360987+0000", 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360988+0000", 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.850000000000001e-07 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.360995+0000", 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5120000000000002e-06 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361004+0000", 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361013+0000", 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4440000000000008e-06 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362000+0000", 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098763200000000001 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362055+0000", 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4580000000000003e-05 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362399+0000", 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034424899999999998 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.363048+0000", 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00064926700000000003 2024-09-17T17:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364011+0000", 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096274900000000005 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364074+0000", 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2903000000000004e-05 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:731 3.es0 3:74f6cb56:::smithi10238019-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.214737112000002, 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039696050000000002, 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 731 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361025+0000", 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361026+0000", 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361028+0000", 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.678e-06 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361035+0000", 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9829999999999999e-06 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.361041+0000", 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362876+0000", 2024-09-17T17:30:57.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001835267 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.362939+0000", 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2508000000000005e-05 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364285+0000", 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013463780000000001 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364367+0000", 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2318999999999996e-05 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.364995+0000", 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000627182 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:7 3.es0 3:71a2b96a:::smithi10238019-23:head [assert-version v25,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.214514424000001, 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077714619999999998, 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361248+0000", 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361249+0000", 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.579e-06 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361265+0000", 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5812999999999999e-05 2024-09-17T17:30:57.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361290+0000", 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4666000000000001e-05 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361298+0000", 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1990000000000006e-06 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361353+0000", 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4898999999999997e-05 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.361425+0000", 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1644999999999996e-05 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.369019+0000", 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075946609999999999 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:895 3.es0 3:7a426564:::smithi10238019-31:head [copy-from ver 68] snapc 1f=[1f,1e,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.365975+0000", 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.209787626000001, 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024857400000000002, 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 895 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365975+0000", 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365975+0000", 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365976+0000", 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6500000000000001e-06 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365984+0000", 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.365997+0000", 2024-09-17T17:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3365e-05 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366004+0000", 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4850000000000003e-06 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366071+0000", 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6552999999999998e-05 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366176+0000", 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010516 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383179+0000", 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017002529999999998 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385694+0000", 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025152289999999999 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.390726+0000", 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050323570000000003 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.390832+0000", 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105641 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:896 3.es0 3:7a426564:::smithi10238019-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.366029+0000", 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.209733246000001, 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019356549000000001, 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 896 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366029+0000", 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366029+0000", 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366030+0000", 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366033+0000", 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.247e-06 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366041+0000", 2024-09-17T17:30:57.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3960000000000006e-06 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366048+0000", 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0949999999999998e-06 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366332+0000", 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028421099999999998 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for blocked object", 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.366363+0000", 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0908999999999998e-05 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385136+0000", 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018772481000000001 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385238+0000", 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101969 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.385385+0000", 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014762999999999999 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.es0 100/32 ECSubWriteReply(tid=83, last_complete=100'66, committed=1, applied=1))", 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:51.558320+0000", 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0174417550000001, 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57883568500000004, 2024-09-17T17:30:57.686 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.558320+0000", 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.558320+0000", 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.558321+0000", 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5600000000000004e-07 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.558326+0000", 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3590000000000001e-06 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:52.136610+0000", 2024-09-17T17:30:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57828400400000002 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:52.136622+0000", 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2216e-05 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:52.136712+0000", 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9988000000000006e-05 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:57.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:52.137156+0000", 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044426200000000001 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:57.690 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:57.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:30:57.693 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:57.689+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:57.793 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:57.791+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:57.841 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:57.841 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:57.841 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:30:57.841 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:57.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:30:57.895 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:57.893+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:57.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:30:57.994 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:57.992+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:58.095 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:58.094+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:58.115 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:30:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.429951505000005, 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:30:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.398998282000001, 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:30:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.366696638999997, 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:30:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:30:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.364772137000003, 2024-09-17T17:30:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:30:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:410 3.9s0 3:93d1baed:::smithi10238019-5:head [copy-from ver 8] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.774875942999998, 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017989010999999999, 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 410 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338102+0000", 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5349999999999999e-06 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338107+0000", 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000003e-06 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338121+0000", 2024-09-17T17:30:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3743e-05 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338129+0000", 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4209999999999995e-06 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8885000000000003e-05 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338278+0000", 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0192999999999995e-05 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.349495+0000", 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011217173 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350828+0000", 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013328700000000001 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356035+0000", 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052068000000000001 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356090+0000", 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4731000000000002e-05 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 52/35 ECSubWriteReply(tid=76, last_complete=52'24, committed=1, applied=1))", 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.703424319, 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.18030790999999999, 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:30:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409553+0000", 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409560+0000", 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589756+0000", 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.180196197 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589765+0000", 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589834+0000", 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9301000000000006e-05 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589860+0000", 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6058999999999999e-05 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:479 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~131072] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.764898795000001, 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017167436000000001, 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 479 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348079+0000", 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.719e-06 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348181+0000", 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101258 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348196+0000", 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5423000000000001e-05 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348204+0000", 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348898+0000", 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069432499999999998 2024-09-17T17:30:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349042+0000", 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143848 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349752+0000", 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071013599999999999 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351054+0000", 2024-09-17T17:30:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301618 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365180+0000", 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014125511 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365245+0000", 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5647000000000004e-05 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:480 3.0s0 3:00c2d3b1:::smithi10238019-29:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.764756427000002, 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017175950999999998, 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 480 2024-09-17T17:30:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348221+0000", 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348225+0000", 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969999999999998e-06 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348235+0000", 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312000000000001e-05 2024-09-17T17:30:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348241+0000", 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340000000000002e-06 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350222+0000", 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019816209999999998 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350336+0000", 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113779 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350754+0000", 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041740400000000002 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351686+0000", 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093241600000000004 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365336+0000", 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013650254000000001 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365396+0000", 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9704000000000003e-05 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:481 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.764725345000002, 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018899091, 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 481 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348252+0000", 2024-09-17T17:30:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348255+0000", 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6720000000000002e-06 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348265+0000", 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.132e-06 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348271+0000", 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351236+0000", 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029650309999999999 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351333+0000", 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7449999999999997e-05 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365440+0000", 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106966 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365519+0000", 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8703999999999997e-05 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.367150+0000", 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001631354 2024-09-17T17:30:58.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 68/32 ECSubWrite(tid=191, reqid=client.4566.0:626, at_version=68'97, trim_to=0'0, roll_forward_to=68'94))", 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.612730364000001, 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018870181999999999, 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500247+0000", 2024-09-17T17:30:58.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8299999999999996e-07 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500279+0000", 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2601000000000002e-05 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518513+0000", 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018233269 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518520+0000", 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2590000000000003e-06 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518590+0000", 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518600+0000", 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7329999999999998e-06 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519071+0000", 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047090800000000003 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519116+0000", 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5521e-05 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.164 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.12s1 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.612289328999999, 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84719745400000002, 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500688+0000", 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3e-07 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500724+0000", 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-05 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346201+0000", 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84547703600000002 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346210+0000", 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346289+0000", 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8825999999999994e-05 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346300+0000", 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1602e-05 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347848+0000", 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00154776 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347885+0000", 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6723000000000001e-05 2024-09-17T17:30:58.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:651 3.0s0 3:07ceeeba:::smithi10238019-36:head [setxattr _header (57) in=7b] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.765668448, 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.090354326999999998, 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 651 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:30:58.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347309+0000", 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347313+0000", 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7270000000000001e-06 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347320+0000", 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1019999999999999e-06 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347330+0000", 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0552e-05 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353490+0000", 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061600309999999998 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353716+0000", 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000225987 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354187+0000", 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047076600000000001 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355021+0000", 2024-09-17T17:30:58.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083427900000000005 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437590+0000", 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.082568902999999999 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437662+0000", 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2272000000000002e-05 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:652 3.0s0 3:07ceeeba:::smithi10238019-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.765636397999998, 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.091356190000000004, 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 652 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347341+0000", 2024-09-17T17:30:58.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347344+0000", 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0000000000000001e-06 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347352+0000", 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8250000000000005e-06 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347357+0000", 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6489999999999996e-06 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354533+0000", 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007175781 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354625+0000", 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2064999999999994e-05 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438014+0000", 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.083388730999999994 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438102+0000", 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7861000000000005e-05 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438696+0000", 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059467799999999996 2024-09-17T17:30:58.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 69/67 ECSubWriteReply(tid=135, last_complete=69'58, committed=1, applied=1))", 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.756847491000002, 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081843374999999996, 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356130+0000", 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3600000000000003e-07 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356133+0000", 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.794e-06 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437473+0000", 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081339488000000001 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437480+0000", 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6810000000000001e-06 2024-09-17T17:30:58.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437533+0000", 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2958999999999999e-05 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437972+0000", 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043981700000000001 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 73/72 ECSubReadReply(tid=170, attrs_read=0))", 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.080043795999998, 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.308285174, 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.37e-07 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.033151+0000", 2024-09-17T17:30:58.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000217491 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337311+0000", 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.304160612 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337319+0000", 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4030000000000001e-06 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.340881+0000", 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035619649999999998 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.341218+0000", 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033716600000000002 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 74/72 ECSubReadReply(tid=172, attrs_read=0))", 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:58.187 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.757900362999997, 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64532973699999996, 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355077+0000", 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355304+0000", 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000086+0000", 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64478191900000004 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000092+0000", 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1800000000000001e-06 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000163+0000", 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0883999999999994e-05 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000406+0000", 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024273599999999999 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPushReply(3.9s0 74/72 [PushReplyOp(3:96cbaefd:::smithi10238019-46:head)])", 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.100567722000001, 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32756439999999998, 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012412+0000", 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7250000000000002e-06 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337269+0000", 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32485740099999999 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337283+0000", 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3655999999999999e-05 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339351+0000", 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020684850000000001 2024-09-17T17:30:58.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339973+0000", 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062179200000000003 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 89/75 ECSubWriteReply(tid=200, last_complete=89'71, committed=1, applied=1))", 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.394230755999999, 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91509105899999998, 2024-09-17T17:30:58.195 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718747+0000", 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.243e-06 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633701+0000", 2024-09-17T17:30:58.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91495358800000004 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633709+0000", 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633787+0000", 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7503999999999995e-05 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633837+0000", 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0012000000000002e-05 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:911 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 822626~1028766,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.74023356, 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.94643117499999996, 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:30:58.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 911 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372744+0000", 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6000000000000003e-07 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372746+0000", 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372758+0000", 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999e-05 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372764+0000", 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5289999999999999e-06 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.373482+0000", 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071808600000000005 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.373547+0000", 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5382000000000005e-05 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.319174+0000", 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.94562681400000004 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 97/75 ECSubWriteReply(tid=225, last_complete=97'88, committed=1, applied=1))", 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7382205469999992, 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21796726999999999, 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6800000000000001e-07 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:30:58.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374758+0000", 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7379999999999999e-06 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592512+0000", 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21775348 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592527+0000", 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5659999999999999e-05 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592649+0000", 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121579 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592723+0000", 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4444999999999996e-05 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:30:58.207 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:30:58.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:30:58.209 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:30:58.207+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:30:58.309 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:58.307+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:58.331 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:58.331 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:58.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:30:58.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:30:58.410 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:58.408+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:58.488 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:58.488 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:58.488 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:58.489 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:58.489 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:58.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:30:58.510 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:30:58.509+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:30:58.608 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 70.129053100999997, 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:30:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:30:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 69.610786863000001, 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:30:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:30:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 67.919069339999993, 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:58.619 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.620 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:30:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 67.899626057999996, 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:30:58.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.626 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 67.899239155000004, 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.630 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:30:58.632 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.633 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.634 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 67.899205788000003, 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.635 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.637 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:30:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.640 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 67.894756317000002, 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.641 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:30:58.643 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:367 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~589824] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:58.644 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.267145501999998, 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012183095999999999, 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 367 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:58.645 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338693+0000", 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.795e-06 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.646 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341609+0000", 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029156210000000002 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341623+0000", 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4583000000000001e-05 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341630+0000", 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2230000000000001e-06 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341691+0000", 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0878000000000002e-05 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341771+0000", 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9308999999999996e-05 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343069+0000", 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012979420000000001 2024-09-17T17:30:58.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348577+0000", 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0055080229999999999 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350781+0000", 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002204742 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350874+0000", 2024-09-17T17:30:58.649 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2979999999999994e-05 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:368 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~573440] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.263505485000003, 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0088382370000000005, 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.650 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 368 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:30:58.651 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342332+0000", 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.08e-06 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342959+0000", 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062686600000000001 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.652 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343062+0000", 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010252 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343083+0000", 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1135e-05 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344260+0000", 2024-09-17T17:30:58.653 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011769969999999999 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344389+0000", 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012922399999999999 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345712+0000", 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001323182 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.654 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348773+0000", 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030611649999999998 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351091+0000", 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023171590000000001 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351169+0000", 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8909e-05 2024-09-17T17:30:58.655 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:370 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.262136032999997, 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0082819799999999996, 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.656 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 370 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.657 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3799999999999997e-07 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343710+0000", 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2570000000000007e-06 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343715+0000", 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6829999999999996e-06 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343721+0000", 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349122+0000", 2024-09-17T17:30:58.659 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005400841 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349229+0000", 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107076 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349735+0000", 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050621599999999998 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.660 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351781+0000", 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020453279999999999 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351908+0000", 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012790499999999999 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351983+0000", 2024-09-17T17:30:58.661 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4154999999999997e-05 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:371 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.262108990000002, 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010270708, 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 371 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.663 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4799999999999999e-07 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343730+0000", 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4389999999999999e-06 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.664 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343735+0000", 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1919999999999998e-06 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343738+0000", 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1360000000000001e-06 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352473+0000", 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0087356429999999995 2024-09-17T17:30:58.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352570+0000", 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6261999999999994e-05 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353998+0000", 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001428688 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.fs6 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 51.225458746999998, 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75327243099999996, 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.667 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380379+0000", 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8100000000000002e-07 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380396+0000", 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7717e-05 2024-09-17T17:30:58.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132975+0000", 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75257808000000004 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132981+0000", 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2249999999999997e-06 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133046+0000", 2024-09-17T17:30:58.669 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4944999999999996e-05 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133053+0000", 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133623+0000", 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057078100000000002 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133650+0000", 2024-09-17T17:30:58.670 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6996e-05 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:495 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 49.947772618000002, 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.022462405000000001, 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:30:58.671 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 495 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:30:58.672 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658065+0000", 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2600000000000001e-07 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658072+0000", 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8430000000000004e-06 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.673 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658082+0000", 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.927e-06 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658089+0000", 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8909999999999998e-06 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660406+0000", 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023178890000000001 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660504+0000", 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7341000000000002e-05 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661132+0000", 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062810899999999996 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.675 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661861+0000", 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072939299999999995 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680403+0000", 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018541588000000001 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680526+0000", 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012359799999999999 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.676 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/35 ECSubWriteReply(tid=37, last_complete=57'27, committed=1, applied=1))", 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 49.944005791999999, 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019253007999999999, 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.677 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661832+0000", 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:30:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661839+0000", 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7999999999999999e-06 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680112+0000", 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018272335000000001 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.679 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680126+0000", 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3867e-05 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680275+0000", 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000149741 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.681084+0000", 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00080847500000000001 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.680 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=196, reqid=client.4566.0:639, at_version=68'102, trim_to=0'0, roll_forward_to=68'97))", 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.080100354999999, 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.667795892, 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.681 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525737+0000", 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:30:58.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525756+0000", 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9026999999999999e-05 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192770+0000", 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.66701338300000002 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192793+0000", 2024-09-17T17:30:58.683 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3203000000000001e-05 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192903+0000", 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109973 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192912+0000", 2024-09-17T17:30:58.684 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9139999999999997e-06 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193502+0000", 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058984700000000001 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193532+0000", 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0651999999999997e-05 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=197, reqid=client.4566.0:639, at_version=0'0, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 38.411927857000002, 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15173107799999999, 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.686 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5799999999999999e-07 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.687 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193938+0000", 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8988000000000001e-05 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344048+0000", 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15010903 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344059+0000", 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1148000000000001e-05 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344595+0000", 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00053675200000000004 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344608+0000", 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2211e-05 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345601+0000", 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00099356800000000001 2024-09-17T17:30:58.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345640+0000", 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8723000000000002e-05 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "replica_scrub(pg: 3.16s4,from:0'0,to:83'146,epoch:84/74,start:3:68000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.68417672, 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.13087489099999999, 2024-09-17T17:30:58.690 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921661+0000", 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1230000000000001e-06 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921668+0000", 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5470000000000002e-06 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052265+0000", 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.130597518 2024-09-17T17:30:58.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052281+0000", 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5537e-05 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052444+0000", 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000163454 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.693 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052471+0000", 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6979999999999999e-05 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052535+0000", 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3732000000000006e-05 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 91/32 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=91'144, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:58.694 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345231+0000", 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.260605282, 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0094121650000000001, 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345231+0000", 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.695 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345231+0000", 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345232+0000", 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2100000000000001e-07 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345260+0000", 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8119e-05 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345309+0000", 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9007e-05 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345319+0000", 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0522e-05 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.347887+0000", 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002567897 2024-09-17T17:30:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.347905+0000", 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7796e-05 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354579+0000", 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0066740000000000002 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.698 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354644+0000", 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4603000000000005e-05 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.346888+0000", 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.258948216, 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010283685000000001, 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.699 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346888+0000", 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346888+0000", 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346889+0000", 2024-09-17T17:30:58.700 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6700000000000002e-07 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346916+0000", 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6548999999999999e-05 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346966+0000", 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0084999999999999e-05 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.701 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346976+0000", 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.859e-06 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348683+0000", 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001707124 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348695+0000", 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2055000000000001e-05 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.702 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.357139+0000", 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0084439010000000002 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.357172+0000", 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3145000000000002e-05 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.703 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/94 ECSubWriteReply(tid=94, last_complete=95'60, committed=1, applied=1))", 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:46.358704+0000", 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 12.247132252, 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014087463, 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358704+0000", 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358704+0000", 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358705+0000", 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8899999999999998e-07 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358711+0000", 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1210000000000003e-06 2024-09-17T17:30:58.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.371995+0000", 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.013283483 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.372008+0000", 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.344e-05 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:58.706 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.372159+0000", 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00015027399999999999 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.372792+0000", 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00063325600000000001 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:58.707 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:58.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:30:58.709 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:30:58.707+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:30:58.810 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:58.808+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:58.816 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:58.817 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:58.817 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:58.817 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:58.825 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:30:58.911 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:58.910+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:58.932 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:58.932 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:58.933 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:58.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:58.933 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:58.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:30:59.012 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:30:59.010+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:30:59.049 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 70.055451742000002, 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.051 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:30:59.052 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.053 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:30:59.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 65.321378702000004, 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.055 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.056 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.058 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:30:59.059 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 64.353350829999997, 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.060 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.061 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.062 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:30:59.063 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.064 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 64.353108719999994, 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:30:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.066 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:30:59.067 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.068 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:30:59.069 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.070 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 50.662616903, 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029370899999999998, 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384601+0000", 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8e-07 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385325+0000", 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072435200000000005 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385406+0000", 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0667999999999994e-05 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385415+0000", 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385509+0000", 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4137999999999999e-05 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385519+0000", 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0208e-05 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387496+0000", 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001976387 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387538+0000", 2024-09-17T17:30:59.075 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1996999999999998e-05 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 50.661888740999999, 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029662, 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.076 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385362+0000", 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2403999999999997e-05 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385471+0000", 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109485 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.078 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385479+0000", 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6750000000000002e-06 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386169+0000", 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068989099999999999 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386179+0000", 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0278000000000001e-05 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388245+0000", 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020665150000000001 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388295+0000", 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9369000000000001e-05 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.608141641000003, 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029678690000000001, 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:59.081 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1899999999999998e-07 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439101+0000", 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4655e-05 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439183+0000", 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2472000000000006e-05 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439193+0000", 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0063e-05 2024-09-17T17:30:59.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439257+0000", 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3775000000000005e-05 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439270+0000", 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2588e-05 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.084 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441995+0000", 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002725505 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442044+0000", 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8392e-05 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 70/67 ECSubWrite(tid=155, reqid=osd.6.0:155, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:59.085 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 37.70204897, 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00308939, 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.086 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345169+0000", 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345192+0000", 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3241e-05 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.087 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345239+0000", 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6607999999999998e-05 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345247+0000", 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5339999999999999e-06 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345711+0000", 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00046389400000000001 2024-09-17T17:30:59.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345720+0000", 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0710000000000001e-06 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348210+0000", 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024892099999999999 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348258+0000", 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8208000000000003e-05 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=199, reqid=osd.11.0:199, at_version=70'104, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 37.701754962999999, 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002830782, 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.090 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345463+0000", 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345486+0000", 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3453999999999999e-05 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.092 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345559+0000", 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2760000000000001e-05 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345586+0000", 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7670000000000001e-05 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345650+0000", 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3139999999999995e-05 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345661+0000", 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1342000000000001e-05 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348252+0000", 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025906829999999999 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348293+0000", 2024-09-17T17:30:59.094 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.155e-05 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=200, reqid=osd.11.0:200, at_version=70'106, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 37.701173529000002, 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027884030000000001, 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346045+0000", 2024-09-17T17:30:59.096 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346081+0000", 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5942999999999998e-05 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346125+0000", 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3949999999999998e-05 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.097 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346134+0000", 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5019999999999998e-06 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346909+0000", 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077504099999999997 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346921+0000", 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1598e-05 2024-09-17T17:30:59.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348789+0000", 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001868268 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348832+0000", 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3059000000000002e-05 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.703816482000001, 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003502827, 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343433+0000", 2024-09-17T17:30:59.101 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2026999999999997e-05 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343478+0000", 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4561999999999998e-05 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343489+0000", 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.102 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343600+0000", 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000110818 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343617+0000", 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6824999999999999e-05 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346851+0000", 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032341750000000002 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346904+0000", 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2850999999999999e-05 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 91/32 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=91'144, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.702376559000001, 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004955597, 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.105 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5300000000000004e-07 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344871+0000", 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0083e-05 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344995+0000", 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012343400000000001 2024-09-17T17:30:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345025+0000", 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0437999999999999e-05 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345098+0000", 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3114999999999994e-05 2024-09-17T17:30:59.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345110+0000", 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2004999999999999e-05 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.108 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349747+0000", 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046367179999999997 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349796+0000", 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9251e-05 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.109 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 91/32 ECSubWrite(tid=101, reqid=osd.7.0:101, at_version=91'83, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.70170006, 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0081096810000000005, 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7499999999999999e-07 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345541+0000", 2024-09-17T17:30:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3065999999999999e-05 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345608+0000", 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7149000000000004e-05 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345617+0000", 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.112 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345692+0000", 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5053000000000001e-05 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345703+0000", 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1172000000000001e-05 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353581+0000", 2024-09-17T17:30:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0078780229999999996 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353627+0000", 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5759e-05 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 91/32 ECSubWrite(tid=102, reqid=osd.7.0:102, at_version=91'85, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.70167399, 2024-09-17T17:30:59.114 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0089760780000000002, 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345544+0000", 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.75e-07 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345564+0000", 2024-09-17T17:30:59.116 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0519000000000001e-05 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348824+0000", 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032595050000000002 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348834+0000", 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0329000000000001e-05 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.117 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349653+0000", 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000818445 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349662+0000", 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5119999999999997e-06 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354468+0000", 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004805363 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.118 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354519+0000", 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1929999999999999e-05 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.698351291000002, 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003290444, 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.120 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348867+0000", 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348892+0000", 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5429e-05 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348942+0000", 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9731999999999998e-05 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348950+0000", 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3350000000000007e-06 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349267+0000", 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00031690899999999998 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349280+0000", 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2880999999999999e-05 2024-09-17T17:30:59.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352124+0000", 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028441819999999998 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352157+0000", 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2177e-05 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 91/32 ECSubWrite(tid=75, reqid=client.4566.0:895, at_version=91'58, trim_to=0'0, roll_forward_to=83'57))", 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 16.663814009999999, 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0040288700000000004, 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:30:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383404+0000", 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6399999999999998e-07 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385486+0000", 2024-09-17T17:30:59.125 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020826339999999999 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385523+0000", 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6739999999999997e-05 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385534+0000", 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0937e-05 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.126 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385626+0000", 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2428000000000003e-05 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385655+0000", 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8343e-05 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.387395+0000", 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001740157 2024-09-17T17:30:59.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.387432+0000", 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7267000000000001e-05 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 92/32 ECSubWrite(tid=282, reqid=osd.11.0:282, at_version=92'148, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 15.697241726, 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029263319999999998, 2024-09-17T17:30:59.128 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4999999999999999e-07 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350009+0000", 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2889999999999999e-05 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350288+0000", 2024-09-17T17:30:59.130 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00027886300000000002 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350299+0000", 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1931e-05 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350372+0000", 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2140999999999999e-05 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.131 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350386+0000", 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4194e-05 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352851+0000", 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024656330000000001 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352902+0000", 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0529999999999999e-05 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 92/32 ECSubWrite(tid=283, reqid=osd.11.0:283, at_version=92'150, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 15.697205582, 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029327540000000001, 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.133 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4999999999999998e-07 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.134 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350038+0000", 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5956e-05 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350867+0000", 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00082833600000000005 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350881+0000", 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4865000000000001e-05 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350949+0000", 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7794999999999997e-05 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350960+0000", 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1083e-05 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352909+0000", 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019484750000000001 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352945+0000", 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5794000000000003e-05 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 10.669248823, 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0037244769999999999, 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6499999999999999e-07 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378414+0000", 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044483400000000002 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378460+0000", 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5980999999999999e-05 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378468+0000", 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3049999999999993e-06 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378582+0000", 2024-09-17T17:30:59.140 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000114036 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378597+0000", 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4647000000000001e-05 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.381637+0000", 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003039698 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.381693+0000", 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6410999999999999e-05 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.18s4 100/75 ECSubWrite(tid=176, reqid=client.4566.0:964, at_version=100'31, trim_to=0'0, roll_forward_to=76'29))", 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:51.788648+0000", 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 7.2585691030000001, 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0054892819999999998, 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:30:59.142 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.788648+0000", 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.788648+0000", 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.788649+0000", 2024-09-17T17:30:59.143 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5499999999999997e-07 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792114+0000", 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0034652390000000002 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792139+0000", 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4669999999999999e-05 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:30:59.144 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792145+0000", 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8209999999999998e-06 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792202+0000", 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7556999999999998e-05 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792209+0000", 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.794100+0000", 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001891085 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.794138+0000", 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7149e-05 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:30:59.146 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:59.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:30:59.148 INFO:tasks.ceph.osd.13.smithi026.stderr:2024-09-17T17:30:59.146+0000 7f09c5849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 33235) UID: 0 2024-09-17T17:30:59.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T17:30:59.249 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:59.247+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:59.295 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:59.295 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:59.295 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:30:59.295 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:59.299 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:30:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:30:59.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:30:59.321 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:30:59.322 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:30:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:30:59.344 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:30:59.349 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-09-17T17:30:59.347+0000 7f8381e49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33390) UID: 0 2024-09-17T17:30:59.366 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:30:59.366 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:30:59.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:30:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:30:59.408 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:30:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:30:59.413 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:30:59.413 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:30:59.413 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:30:59.413 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:30:59.413 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:30:59.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:30:59.431 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:30:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:30:59.452 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:30:59.449+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:30:59.453 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:30:59.453 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:30:59.473 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:30:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:30:59.494 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:30:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:30:59.516 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:30:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:30:59.529 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:59.529 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:59.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:30:59.539 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:30:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:30:59.559 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:30:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:30:59.580 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:30:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:30:59.583 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:30:59.581+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:30:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:30:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:59.653 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:30:59.651+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:30:59.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:30:59.754 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:30:59.753+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:30:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:59.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:30:59.855 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:30:59.853+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:30:59.920 INFO:teuthology.orchestra.run.smithi026.stdout:ERROR: (22) Invalid argument 2024-09-17T17:30:59.920 INFO:teuthology.orchestra.run.smithi026.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:30:59.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:30:59.941 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:30:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-17T17:30:59.956 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:30:59.954+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:30:59.963 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:30:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:31:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.049 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.057 INFO:tasks.ceph.osd.15.smithi102.stderr:2024-09-17T17:31:00.055+0000 7f2f7cc52640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33295) UID: 0 2024-09-17T17:31:00.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:31:00.157 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:31:00.155+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:31:00.169 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.169 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:31:00.258 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:31:00.257+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:31:00.290 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.290 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:31:00.359 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:31:00.358+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:31:00.418 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.418 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:31:00.461 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:31:00.459+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:31:00.562 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:31:00.561+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:31:00.577 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.578 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:31:00.663 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:31:00.661+0000 7fee917d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33389) UID: 0 2024-09-17T17:31:00.734 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.734 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:31:00.764 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:31:00.762+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:31:00.850 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.850 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:31:00.864 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:31:00.863+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:31:00.965 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:00.965 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:00.965 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:31:00.963+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:31:00.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:31:01.066 INFO:tasks.ceph.osd.14.smithi027.stderr:2024-09-17T17:31:01.064+0000 7fb1b2e60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 33947) UID: 0 2024-09-17T17:31:01.082 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.082 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:31:01.167 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:31:01.165+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:31:01.238 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.238 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:31:01.268 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:31:01.266+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:31:01.353 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.354 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:31:01.369 INFO:tasks.ceph.osd.11.smithi102.stderr:2024-09-17T17:31:01.368+0000 7f4e66fee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33293) UID: 0 2024-09-17T17:31:01.468 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.469 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.470 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:31:01.468+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:31:01.478 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:31:01.570 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:31:01.569+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:31:01.591 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.591 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.600 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:31:01.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:31:01.671 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T17:31:01.669+0000 7fafb3abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33384) UID: 0 2024-09-17T17:31:01.704 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.704 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.712 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:31:01.772 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:31:01.770+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:31:01.816 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.816 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.825 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:31:01.909 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:31:01.907+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:31:01.967 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:01.968 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:01.974 INFO:tasks.ceph.osd.12.smithi007.stderr:2024-09-17T17:31:01.972+0000 7fa04e8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 33393) UID: 0 2024-09-17T17:31:01.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T17:31:01.976 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:31:02.074 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:31:02.073+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:31:02.109 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T17:31:02.110 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T17:31:02.120 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:02.120 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:02.129 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:31:02.132 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T17:31:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T17:31:02.154 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T17:31:02.154 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T17:31:02.175 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:31:02.173+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T17:31:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T17:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T17:31:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T17:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T17:31:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T17:31:02.231 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:02.231 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:02.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-17T17:31:02.241 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T17:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-17T17:31:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T17:31:02.276 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-17T17:31:02.274+0000 7fe65c513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33229) UID: 0 2024-09-17T17:31:02.283 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8: {} 2024-09-17T17:31:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-17T17:31:02.306 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9: {} 2024-09-17T17:31:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-17T17:31:02.328 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10: {} 2024-09-17T17:31:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-17T17:31:02.342 INFO:teuthology.orchestra.run.smithi102.stdout:ERROR: (22) Invalid argument 2024-09-17T17:31:02.342 INFO:teuthology.orchestra.run.smithi102.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T17:31:02.349 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11: {} 2024-09-17T17:31:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-17T17:31:02.351 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:31:02.370 INFO:teuthology.orchestra.run.smithi007.stdout:osd.12: {} 2024-09-17T17:31:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-17T17:31:02.377 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:31:02.375+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:31:02.393 INFO:teuthology.orchestra.run.smithi007.stdout:osd.13: {} 2024-09-17T17:31:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-17T17:31:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:31:02.413+0000 7fe183fff640 0 -- 172.21.15.7:0/4100337516 >> [v2:172.21.15.27:6824/1199123829,v1:172.21.15.27:6825/1199123829] conn(0x7fe18c21f260 msgr2=0x7fe18c221650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-17T17:31:02.421 INFO:teuthology.orchestra.run.smithi007.stdout:osd.14: {} 2024-09-17T17:31:02.443 INFO:teuthology.orchestra.run.smithi007.stdout:osd.15: {} 2024-09-17T17:31:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-17T17:31:02.455 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:02.455 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:02.455 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:31:02.455 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:31:02.456 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:02.464 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:31:02.507 INFO:tasks.ceph.osd.7.smithi102.stderr:2024-09-17T17:31:02.506+0000 7f1b645a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33288) UID: 0 2024-09-17T17:31:02.568 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:02.568 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=24, last_complete=36'10, committed=1, applied=1))", 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 74.111017574000002, 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53983539199999997, 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.569 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.570 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454674+0000", 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0790000000000002e-06 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.571 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992637+0000", 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53796236600000003 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992644+0000", 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5820000000000004e-06 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.572 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994469+0000", 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018243350000000001 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994505+0000", 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6300999999999997e-05 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.573 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:174 3.cs0 3:30756a83:::smithi10238019-35:2 [read 501714~71726,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 73.575549480999996, 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011875784, 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 174 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.574 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9400000000000005e-07 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990142+0000", 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.557e-06 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990195+0000", 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3012e-05 2024-09-17T17:31:02.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990206+0000", 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0876000000000001e-05 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990783+0000", 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00057770200000000003 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.577 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990850+0000", 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6168999999999996e-05 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.002014+0000", 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011163874000000001 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.578 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:175 3.cs0 3:30756a83:::smithi10238019-35:2 [read 194830~378610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 73.575462923000003, 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.30022850299999998, 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 175 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990225+0000", 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8199999999999998e-07 2024-09-17T17:31:02.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990230+0000", 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7809999999999996e-06 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990371+0000", 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014071299999999999 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990381+0000", 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0061000000000001e-05 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994702+0000", 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043214899999999999 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994769+0000", 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6600999999999995e-05 2024-09-17T17:31:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290453+0000", 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.29568397499999999 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=28, attrs_read=0))", 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 73.564489906999995, 2024-09-17T17:31:02.583 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.289267987, 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:31:02.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001198+0000", 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6000000000000004e-07 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001279+0000", 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1168000000000006e-05 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289843+0000", 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.28856357599999999 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289854+0000", 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1749999999999999e-05 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289975+0000", 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012060100000000001 2024-09-17T17:31:02.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290465+0000", 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049033200000000001 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:316 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (56) in=7b] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:31:02.587 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.576229142000003, 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041035350999999998, 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 316 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.588 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989459+0000", 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2200000000000004e-07 2024-09-17T17:31:02.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989462+0000", 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.287e-06 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989469+0000", 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7229999999999998e-06 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989472+0000", 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.428e-06 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990430+0000", 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00095784099999999997 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990525+0000", 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5129000000000003e-05 2024-09-17T17:31:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990951+0000", 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042552399999999999 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991309+0000", 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000358632 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.592 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030441+0000", 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039131753999999998 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030493+0000", 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2510999999999999e-05 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.593 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:317 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.576195806000001, 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041694146000000001, 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 317 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.594 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989492+0000", 2024-09-17T17:31:02.595 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3099999999999998e-07 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989494+0000", 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3329999999999999e-06 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989501+0000", 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.516e-06 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989505+0000", 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.912e-06 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991167+0000", 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001662188 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991214+0000", 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6944000000000003e-05 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030733+0000", 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039519406999999999 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030782+0000", 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9338000000000001e-05 2024-09-17T17:31:02.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.031186+0000", 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000403077 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:429 3.cs0 3:30d62bba:::smithi10238019-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:31:02.599 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 59.172784276000002, 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014137011, 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 429 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.600 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:02.601 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3299999999999998e-07 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392905+0000", 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6869999999999999e-06 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392912+0000", 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6250000000000001e-06 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392916+0000", 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7330000000000003e-06 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397110+0000", 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041939450000000001 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397209+0000", 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8961000000000003e-05 2024-09-17T17:31:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402765+0000", 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055556420000000004 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402840+0000", 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4943000000000004e-05 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.407040+0000", 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042001419999999996 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:436 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 12] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:02.605 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 59.172687377999999, 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012854290000000001, 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 436 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.606 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3700000000000001e-07 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.607 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393005+0000", 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4190000000000004e-06 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393008+0000", 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7009999999999998e-06 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393023+0000", 2024-09-17T17:31:02.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4545999999999999e-05 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397903+0000", 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048805719999999997 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397964+0000", 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0520000000000003e-05 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.401036+0000", 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030720470000000001 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402722+0000", 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001685513 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405761+0000", 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030398790000000001 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405854+0000", 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2756000000000004e-05 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:486 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 55.186680885000001, 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75510421100000003, 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.611 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 486 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379007+0000", 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379011+0000", 2024-09-17T17:31:02.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0720000000000001e-06 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379019+0000", 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9580000000000007e-06 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379025+0000", 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6799999999999998e-06 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379992+0000", 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096721299999999997 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380044+0000", 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2091e-05 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380358+0000", 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000313543 2024-09-17T17:31:02.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380731+0000", 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00037292 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134034+0000", 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75330350700000004 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134111+0000", 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6497999999999998e-05 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:487 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 55.186651744000002, 2024-09-17T17:31:02.617 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.756014517, 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 487 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379036+0000", 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5499999999999998e-07 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379038+0000", 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379045+0000", 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8770000000000004e-06 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379050+0000", 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2299999999999999e-06 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380522+0000", 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001471909 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380575+0000", 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2821999999999997e-05 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134417+0000", 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75384171300000002 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134482+0000", 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5789000000000003e-05 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.135050+0000", 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00056751999999999996 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.622 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:626 3.cs0 3:30756a83:::smithi10238019-35:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.067389906999999, 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021079339999999998, 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.623 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 626 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:31:02.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498298+0000", 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498301+0000", 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7530000000000002e-06 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498307+0000", 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9869999999999996e-06 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498310+0000", 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1259999999999998e-06 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499848+0000", 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001537912 2024-09-17T17:31:02.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499903+0000", 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5763000000000001e-05 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500222+0000", 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00031825499999999998 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500848+0000", 2024-09-17T17:31:02.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062612400000000004 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519301+0000", 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018453431999999999 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519377+0000", 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5405000000000001e-05 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.628 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:627 3.cs0 3:30756a83:::smithi10238019-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.067360747000002, 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021540693, 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.629 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 627 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:31:02.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498327+0000", 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.51e-07 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498329+0000", 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3989999999999998e-06 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498337+0000", 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5379999999999997e-06 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498341+0000", 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4349999999999999e-06 2024-09-17T17:31:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500569+0000", 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002227934 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500629+0000", 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9721e-05 2024-09-17T17:31:02.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519409+0000", 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018780192000000001 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519465+0000", 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6062e-05 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519867+0000", 2024-09-17T17:31:02.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000401961 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s6 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.065074441999997, 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26792300099999999, 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:02.635 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500614+0000", 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6700000000000002e-07 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500747+0000", 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013382499999999999 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767249+0000", 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26650138800000001 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767261+0000", 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1785e-05 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767372+0000", 2024-09-17T17:31:02.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111743 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767387+0000", 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4548e-05 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768494+0000", 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011074990000000001 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768536+0000", 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1346e-05 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:639 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.041892435999998, 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67002614400000005, 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.640 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 639 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.641 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0500000000000003e-07 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1729999999999999e-06 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523810+0000", 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1627000000000001e-05 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523815+0000", 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0420000000000002e-06 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525161+0000", 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0013460169999999999 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525253+0000", 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1793999999999996e-05 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526004+0000", 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00075112400000000004 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527034+0000", 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001029713 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193749+0000", 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.66671533999999999 2024-09-17T17:31:02.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193821+0000", 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1709000000000001e-05 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:640 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.041860819999997, 2024-09-17T17:31:02.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67078859899999999, 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 640 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523827+0000", 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.03e-07 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523829+0000", 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1739999999999999e-06 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523836+0000", 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3869999999999997e-06 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523841+0000", 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4089999999999996e-06 2024-09-17T17:31:02.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527242+0000", 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003401383 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527298+0000", 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5705000000000003e-05 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194159+0000", 2024-09-17T17:31:02.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.666861484 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194218+0000", 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9039000000000002e-05 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194615+0000", 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039661500000000001 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.651 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 68/32 ECSubWrite(tid=118, reqid=client.4566.0:641, at_version=68'91, trim_to=0'0, roll_forward_to=66'89))", 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.041019704, 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.164368816, 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:02.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524694+0000", 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6256000000000001e-05 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687716+0000", 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.163021575 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687725+0000", 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9390000000000003e-06 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687798+0000", 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900999999999999e-05 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687809+0000", 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.153e-05 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689007+0000", 2024-09-17T17:31:02.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001197844 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689036+0000", 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9342000000000001e-05 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 78/32 ECSubWrite(tid=154, reqid=osd.3.0:154, at_version=78'120, trim_to=0'0, roll_forward_to=78'118))", 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:31:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.309302354000003, 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.088487576999999998, 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.657 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256386+0000", 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3900000000000004e-07 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256409+0000", 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3475000000000001e-05 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342362+0000", 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.085952671999999994 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342372+0000", 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0285000000000001e-05 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343593+0000", 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001220858 2024-09-17T17:31:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343604+0000", 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1078e-05 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344835+0000", 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0012308309999999999 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344872+0000", 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7539000000000001e-05 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 91/87 ECSubWrite(tid=84, reqid=osd.14.0:84, at_version=91'76, trim_to=0'0, roll_forward_to=89'72))", 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 20.211660538, 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.14743911800000001, 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354048+0000", 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0214999999999999e-05 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500785+0000", 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.146736905 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500799+0000", 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4887e-05 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500878+0000", 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8880000000000004e-05 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500886+0000", 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5270000000000001e-06 2024-09-17T17:31:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.501437+0000", 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000551134 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.501466+0000", 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8836e-05 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:941 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 75] snapc 21=[21,20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 14.203612356000001, 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.020459694, 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 941 2024-09-17T17:31:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362077+0000", 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.984e-06 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362089+0000", 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2456000000000001e-05 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362107+0000", 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8053000000000001e-05 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362119+0000", 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2055999999999999e-05 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362183+0000", 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3246999999999999e-05 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:31:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2470000000000002e-06 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364142+0000", 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0019523780000000001 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364249+0000", 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010689899999999999 2024-09-17T17:31:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.377910+0000", 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013660960999999999 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.380133+0000", 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0022232409999999999 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.382430+0000", 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00229707 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.382535+0000", 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000104102 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:942 3.cs0 3:349443fd:::smithi10238019-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 14.203543722999999, 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.017198940999999999, 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 942 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.96e-07 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362147+0000", 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5219999999999999e-06 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362153+0000", 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8159999999999999e-06 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362157+0000", 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.245e-06 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362251+0000", 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3163000000000002e-05 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362255+0000", 2024-09-17T17:31:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1849999999999997e-06 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364403+0000", 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002147726 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364448+0000", 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5776999999999999e-05 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379154+0000", 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014705371 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379256+0000", 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000102272 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379342+0000", 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6468e-05 2024-09-17T17:31:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:02.679 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:31:02.681 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:31:02.682 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:31:02.679+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:31:02.693 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 2] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:31:02.693 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T17:31:02.694 INFO:tasks.thrashosds.thrasher:Killing osd 8, live_osds are [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-17T17:31:02.694 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-17T17:31:02.694 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:31:02.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:31:02.728 INFO:tasks.ceph.osd.8:Stopped 2024-09-17T17:31:02.729 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.8 2024-09-17T17:31:02.729 INFO:tasks.thrashosds.thrasher:remote for osd 8 is ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:31:02.729 DEBUG:teuthology.orchestra.run.smithi007:> type ceph-objectstore-tool 2024-09-17T17:31:02.781 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:31:02.779+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:31:02.785 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T17:31:02.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-8 --journal-path /var/lib/ceph/osd/ceph-8/journal --op list-pgs 2024-09-17T17:31:02.825 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:31:02.825 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:02.834 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:31:02.882 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-17T17:31:02.881+0000 7fdc718a6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33940) UID: 0 2024-09-17T17:31:02.937 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:31:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:31:02.937 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:02.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:31:02.990 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-17T17:31:02.989+0000 7f9b888d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33287) UID: 0 2024-09-17T17:31:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:31:03.035 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:31:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-17T17:31:02.604292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":122995251,"num_objects":113,"num_object_clones":61,"num_object_copies":771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":17,"num_read":306,"num_read_kb":77852,"num_write":581,"num_write_kb":87506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":42123264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1855,"ondisk_log_size":1855,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":826896,"kb_used_data":275088,"kb_used_omap":24,"kb_used_meta":551399,"kb_avail":1509122544,"statfs":{"total":1546188226560,"available":1545341485056,"internally_reserved":0,"allocated":281690112,"data_stored":388758801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":564632655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016973"},"pg_stats":[{"pgid":"3.19","version":"65'44","reported_seq":418,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T17:30:58.874058+0000","last_change":"2024-09-17T17:30:51.537374+0000","last_active":"2024-09-17T17:30:58.874058+0000","last_peered":"2024-09-17T17:30:58.874058+0000","last_clean":"2024-09-17T17:30:58.874058+0000","last_became_active":"2024-09-17T17:30:26.345748+0000","last_became_peered":"2024-09-17T17:30:26.345748+0000","last_unstale":"2024-09-17T17:30:58.874058+0000","last_undegraded":"2024-09-17T17:30:58.874058+0000","last_fullsized":"2024-09-17T17:30:58.874058+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":5,"last_scrub":"65'44","last_scrub_stamp":"2024-09-17T17:30:27.911326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.911326+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.721701+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00059301299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.18","version":"100'31","reported_seq":431,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.584313+0000","last_change":"2024-09-17T17:30:51.551323+0000","last_active":"2024-09-17T17:30:57.584313+0000","last_peered":"2024-09-17T17:30:57.584313+0000","last_clean":"2024-09-17T17:30:57.584313+0000","last_became_active":"2024-09-17T17:30:27.354929+0000","last_became_peered":"2024-09-17T17:30:27.354929+0000","last_unstale":"2024-09-17T17:30:57.584313+0000","last_undegraded":"2024-09-17T17:30:57.584313+0000","last_fullsized":"2024-09-17T17:30:57.584313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":5,"last_scrub":"76'29","last_scrub_stamp":"2024-09-17T17:30:28.377344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.377344+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:36.001747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0050076549999999997,"stat_sum":{"num_bytes":5963776,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":625,"num_write":1,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.17","version":"40'6","reported_seq":363,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.581254+0000","last_change":"2024-09-17T17:30:51.542436+0000","last_active":"2024-09-17T17:30:57.581254+0000","last_peered":"2024-09-17T17:30:57.581254+0000","last_clean":"2024-09-17T17:30:57.581254+0000","last_became_active":"2024-09-17T17:30:19.460414+0000","last_became_peered":"2024-09-17T17:30:19.460414+0000","last_unstale":"2024-09-17T17:30:57.581254+0000","last_undegraded":"2024-09-17T17:30:57.581254+0000","last_fullsized":"2024-09-17T17:30:57.581254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:25.845481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.845481+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:31.818628+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013824040000000001,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.16","version":"100'181","reported_seq":614,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582321+0000","last_change":"2024-09-17T17:30:51.569089+0000","last_active":"2024-09-17T17:30:57.582321+0000","last_peered":"2024-09-17T17:30:57.582321+0000","last_clean":"2024-09-17T17:30:57.582321+0000","last_became_active":"2024-09-17T17:30:47.363797+0000","last_became_peered":"2024-09-17T17:30:47.363797+0000","last_unstale":"2024-09-17T17:30:57.582321+0000","last_undegraded":"2024-09-17T17:30:57.582321+0000","last_fullsized":"2024-09-17T17:30:57.582321+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'146","last_scrub_stamp":"2024-09-17T17:30:36.055244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.055244+0000","objects_scrubbed":7,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:05.906343+0000","scrub_duration":133,"objects_trimmed":0,"snaptrim_duration":0.028782753000000001,"stat_sum":{"num_bytes":13795328,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":4,"num_read_kb":3,"num_write":13,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.15","version":"100'41","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.578622+0000","last_change":"2024-09-17T17:30:56.350554+0000","last_active":"2024-09-17T17:30:57.578622+0000","last_peered":"2024-09-17T17:30:57.578622+0000","last_clean":"2024-09-17T17:30:57.578622+0000","last_became_active":"2024-09-17T17:30:56.349441+0000","last_became_peered":"2024-09-17T17:30:56.349441+0000","last_unstale":"2024-09-17T17:30:57.578622+0000","last_undegraded":"2024-09-17T17:30:57.578622+0000","last_fullsized":"2024-09-17T17:30:57.578622+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'20","last_scrub_stamp":"2024-09-17T17:30:27.352233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.352233+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:56.369131+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.020157329000000002,"stat_sum":{"num_bytes":2080768,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":11,"num_read_kb":1251,"num_write":4,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3817472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0,3,10,4,5,12],"acting":[14,0,3,10,4,5,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.14","version":"62'42","reported_seq":39,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582420+0000","last_change":"2024-09-17T17:30:54.558673+0000","last_active":"2024-09-17T17:30:57.582420+0000","last_peered":"2024-09-17T17:30:57.582420+0000","last_clean":"2024-09-17T17:30:57.582420+0000","last_became_active":"2024-09-17T17:30:54.556357+0000","last_became_peered":"2024-09-17T17:30:54.556357+0000","last_unstale":"2024-09-17T17:30:57.582420+0000","last_undegraded":"2024-09-17T17:30:57.582420+0000","last_fullsized":"2024-09-17T17:30:57.582420+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":5,"last_scrub":"62'42","last_scrub_stamp":"2024-09-17T17:30:26.686769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.686769+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:37.716688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000950434,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,15,10,7,2,13],"acting":[3,14,15,10,7,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":175,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.658574+0000","last_change":"2024-09-17T17:29:34.269494+0000","last_active":"2024-09-17T17:30:57.658574+0000","last_peered":"2024-09-17T17:30:57.658574+0000","last_clean":"2024-09-17T17:30:57.658574+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:30:57.658574+0000","last_undegraded":"2024-09-17T17:30:57.658574+0000","last_fullsized":"2024-09-17T17:30:57.658574+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:46.115044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"100'151","reported_seq":558,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582451+0000","last_change":"2024-09-17T17:30:51.570498+0000","last_active":"2024-09-17T17:30:57.582451+0000","last_peered":"2024-09-17T17:30:57.582451+0000","last_clean":"2024-09-17T17:30:57.582451+0000","last_became_active":"2024-09-17T17:30:47.362331+0000","last_became_peered":"2024-09-17T17:30:47.362331+0000","last_unstale":"2024-09-17T17:30:57.582451+0000","last_undegraded":"2024-09-17T17:30:57.582451+0000","last_fullsized":"2024-09-17T17:30:57.582451+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'127","last_scrub_stamp":"2024-09-17T17:30:27.836715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.836715+0000","objects_scrubbed":5,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:52.520132+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.030063706999999999,"stat_sum":{"num_bytes":4784128,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":7,"num_read_kb":669,"num_write":7,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":174,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.583049+0000","last_change":"2024-09-17T17:29:33.592849+0000","last_active":"2024-09-17T17:30:57.583049+0000","last_peered":"2024-09-17T17:30:57.583049+0000","last_clean":"2024-09-17T17:30:57.583049+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:30:57.583049+0000","last_undegraded":"2024-09-17T17:30:57.583049+0000","last_fullsized":"2024-09-17T17:30:57.583049+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:31.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"40'6","reported_seq":356,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.581660+0000","last_change":"2024-09-17T17:30:51.544193+0000","last_active":"2024-09-17T17:30:57.581660+0000","last_peered":"2024-09-17T17:30:57.581660+0000","last_clean":"2024-09-17T17:30:57.581660+0000","last_became_active":"2024-09-17T17:30:19.457663+0000","last_became_peered":"2024-09-17T17:30:19.457663+0000","last_unstale":"2024-09-17T17:30:57.581660+0000","last_undegraded":"2024-09-17T17:30:57.581660+0000","last_fullsized":"2024-09-17T17:30:57.581660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-09-17T17:30:24.345150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.345150+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:40.665076+0000","scrub_duration":541,"objects_trimmed":0,"snaptrim_duration":0.0024415970000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T17:30:58.589981+0000","last_change":"2024-09-17T17:29:34.236756+0000","last_active":"2024-09-17T17:30:58.589981+0000","last_peered":"2024-09-17T17:30:58.589981+0000","last_clean":"2024-09-17T17:30:58.589981+0000","last_became_active":"2024-09-17T17:29:31.673829+0000","last_became_peered":"2024-09-17T17:29:31.673829+0000","last_unstale":"2024-09-17T17:30:58.589981+0000","last_undegraded":"2024-09-17T17:30:58.589981+0000","last_fullsized":"2024-09-17T17:30:58.589981+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028255100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'70","reported_seq":497,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T17:31:00.342159+0000","last_change":"2024-09-17T17:30:51.552576+0000","last_active":"2024-09-17T17:31:00.342159+0000","last_peered":"2024-09-17T17:31:00.342159+0000","last_clean":"2024-09-17T17:31:00.342159+0000","last_became_active":"2024-09-17T17:30:46.352101+0000","last_became_peered":"2024-09-17T17:30:46.352101+0000","last_unstale":"2024-09-17T17:31:00.342159+0000","last_undegraded":"2024-09-17T17:31:00.342159+0000","last_fullsized":"2024-09-17T17:31:00.342159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'48","last_scrub_stamp":"2024-09-17T17:30:27.676614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.676614+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:28.397838+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.015274865,"stat_sum":{"num_bytes":6848512,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13172736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,15,9,2,6,10,13],"acting":[1,15,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T17:31:00.342162+0000","last_change":"2024-09-17T17:30:47.507145+0000","last_active":"2024-09-17T17:31:00.342162+0000","last_peered":"2024-09-17T17:31:00.342162+0000","last_clean":"2024-09-17T17:31:00.342162+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:31:00.342162+0000","last_undegraded":"2024-09-17T17:31:00.342162+0000","last_fullsized":"2024-09-17T17:31:00.342162+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.507074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.507074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.099148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'27","reported_seq":32,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.584699+0000","last_change":"2024-09-17T17:30:55.344824+0000","last_active":"2024-09-17T17:30:57.584699+0000","last_peered":"2024-09-17T17:30:57.584699+0000","last_clean":"2024-09-17T17:30:57.584699+0000","last_became_active":"2024-09-17T17:30:54.556335+0000","last_became_peered":"2024-09-17T17:30:54.556335+0000","last_unstale":"2024-09-17T17:30:57.584699+0000","last_undegraded":"2024-09-17T17:30:57.584699+0000","last_fullsized":"2024-09-17T17:30:57.584699+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'27","last_scrub_stamp":"2024-09-17T17:30:28.947105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.947105+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-17T17:31:25.086938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091031499999999997,"stat_sum":{"num_bytes":2981888,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1685,"num_write":22,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,5,2,7,14,3],"acting":[10,0,5,2,7,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.3","version":"100'91","reported_seq":499,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.584704+0000","last_change":"2024-09-17T17:30:51.559448+0000","last_active":"2024-09-17T17:30:57.584704+0000","last_peered":"2024-09-17T17:30:57.584704+0000","last_clean":"2024-09-17T17:30:57.584704+0000","last_became_active":"2024-09-17T17:30:26.345813+0000","last_became_peered":"2024-09-17T17:30:26.345813+0000","last_unstale":"2024-09-17T17:30:57.584704+0000","last_undegraded":"2024-09-17T17:30:57.584704+0000","last_fullsized":"2024-09-17T17:30:57.584704+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'58","last_scrub_stamp":"2024-09-17T17:30:28.942498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.942498+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:55.401727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.018839663,"stat_sum":{"num_bytes":7880704,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":2592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.2","version":"28'2","reported_seq":177,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582714+0000","last_change":"2024-09-17T17:29:33.592515+0000","last_active":"2024-09-17T17:30:57.582714+0000","last_peered":"2024-09-17T17:30:57.582714+0000","last_clean":"2024-09-17T17:30:57.582714+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:30:57.582714+0000","last_undegraded":"2024-09-17T17:30:57.582714+0000","last_fullsized":"2024-09-17T17:30:57.582714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:29:30.568245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:54.817725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"100'96","reported_seq":496,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T17:30:58.874058+0000","last_change":"2024-09-17T17:30:51.555622+0000","last_active":"2024-09-17T17:30:58.874058+0000","last_peered":"2024-09-17T17:30:58.874058+0000","last_clean":"2024-09-17T17:30:58.874058+0000","last_became_active":"2024-09-17T17:30:27.343750+0000","last_became_peered":"2024-09-17T17:30:27.343750+0000","last_unstale":"2024-09-17T17:30:58.874058+0000","last_undegraded":"2024-09-17T17:30:58.874058+0000","last_fullsized":"2024-09-17T17:30:58.874058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'62","last_scrub_stamp":"2024-09-17T17:30:28.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.961363+0000","objects_scrubbed":3,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:38.115499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.018887095999999999,"stat_sum":{"num_bytes":4014080,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":10,"num_read_kb":6919,"num_write":5,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":183,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.578397+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:30:57.578397+0000","last_peered":"2024-09-17T17:30:57.578397+0000","last_clean":"2024-09-17T17:30:57.578397+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:30:57.578397+0000","last_undegraded":"2024-09-17T17:30:57.578397+0000","last_fullsized":"2024-09-17T17:30:57.578397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"100'55","reported_seq":450,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.579952+0000","last_change":"2024-09-17T17:30:51.555660+0000","last_active":"2024-09-17T17:30:57.579952+0000","last_peered":"2024-09-17T17:30:57.579952+0000","last_clean":"2024-09-17T17:30:57.579952+0000","last_became_active":"2024-09-17T17:30:26.354825+0000","last_became_peered":"2024-09-17T17:30:26.354825+0000","last_unstale":"2024-09-17T17:30:57.579952+0000","last_undegraded":"2024-09-17T17:30:57.579952+0000","last_fullsized":"2024-09-17T17:30:57.579952+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:28.852921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:28.852921+0000","objects_scrubbed":2,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:56.003445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.014426430000000001,"stat_sum":{"num_bytes":3211264,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":8117,"num_write":6,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":183,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582170+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:30:57.582170+0000","last_peered":"2024-09-17T17:30:57.582170+0000","last_clean":"2024-09-17T17:30:57.582170+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:30:57.582170+0000","last_undegraded":"2024-09-17T17:30:57.582170+0000","last_fullsized":"2024-09-17T17:30:57.582170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"63'42","reported_seq":408,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.578543+0000","last_change":"2024-09-17T17:30:51.549246+0000","last_active":"2024-09-17T17:30:57.578543+0000","last_peered":"2024-09-17T17:30:57.578543+0000","last_clean":"2024-09-17T17:30:57.578543+0000","last_became_active":"2024-09-17T17:30:46.353966+0000","last_became_peered":"2024-09-17T17:30:46.353966+0000","last_unstale":"2024-09-17T17:30:57.578543+0000","last_undegraded":"2024-09-17T17:30:57.578543+0000","last_fullsized":"2024-09-17T17:30:57.578543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'42","last_scrub_stamp":"2024-09-17T17:30:25.590903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:25.590903+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:54.863995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966410000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0,3,4,15,10,11],"acting":[14,0,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"2.3","version":"26'1","reported_seq":184,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.578721+0000","last_change":"2024-09-17T17:30:38.649965+0000","last_active":"2024-09-17T17:30:57.578721+0000","last_peered":"2024-09-17T17:30:57.578721+0000","last_clean":"2024-09-17T17:30:57.578721+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:30:57.578721+0000","last_undegraded":"2024-09-17T17:30:57.578721+0000","last_fullsized":"2024-09-17T17:30:57.578721+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:30:38.649873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.649873+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:48.468836+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":318,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.584545+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:30:57.584545+0000","last_peered":"2024-09-17T17:30:57.584545+0000","last_clean":"2024-09-17T17:30:57.584545+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:30:57.584545+0000","last_undegraded":"2024-09-17T17:30:57.584545+0000","last_fullsized":"2024-09-17T17:30:57.584545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"75'29","reported_seq":426,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.584409+0000","last_change":"2024-09-17T17:30:51.554183+0000","last_active":"2024-09-17T17:30:57.584409+0000","last_peered":"2024-09-17T17:30:57.584409+0000","last_clean":"2024-09-17T17:30:57.584409+0000","last_became_active":"2024-09-17T17:30:26.348354+0000","last_became_peered":"2024-09-17T17:30:26.348354+0000","last_unstale":"2024-09-17T17:30:57.584409+0000","last_undegraded":"2024-09-17T17:30:57.584409+0000","last_fullsized":"2024-09-17T17:30:57.584409+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'29","last_scrub_stamp":"2024-09-17T17:30:27.407808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.407808+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.453798+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0079920400000000006,"stat_sum":{"num_bytes":344064,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.9","version":"100'75","reported_seq":495,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T17:30:58.874144+0000","last_change":"2024-09-17T17:30:51.549010+0000","last_active":"2024-09-17T17:30:58.874144+0000","last_peered":"2024-09-17T17:30:58.874144+0000","last_clean":"2024-09-17T17:30:58.874144+0000","last_became_active":"2024-09-17T17:30:28.394697+0000","last_became_peered":"2024-09-17T17:30:28.394697+0000","last_unstale":"2024-09-17T17:30:58.874144+0000","last_undegraded":"2024-09-17T17:30:58.874144+0000","last_fullsized":"2024-09-17T17:30:58.874144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'61","last_scrub_stamp":"2024-09-17T17:30:36.024221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:36.024221+0000","objects_scrubbed":4,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:44.846851+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.012117802,"stat_sum":{"num_bytes":9404416,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":5,"num_read_kb":12170,"num_write":2,"num_write_kb":2336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.a","version":"100'140","reported_seq":493,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582652+0000","last_change":"2024-09-17T17:30:51.809656+0000","last_active":"2024-09-17T17:30:57.582652+0000","last_peered":"2024-09-17T17:30:57.582652+0000","last_clean":"2024-09-17T17:30:57.582652+0000","last_became_active":"2024-09-17T17:29:44.656425+0000","last_became_peered":"2024-09-17T17:29:44.656425+0000","last_unstale":"2024-09-17T17:30:57.582652+0000","last_undegraded":"2024-09-17T17:30:57.582652+0000","last_fullsized":"2024-09-17T17:30:57.582652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'140","last_scrub_stamp":"2024-09-17T17:30:51.809533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.809533+0000","objects_scrubbed":8,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:54.076947+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.017590710999999998,"stat_sum":{"num_bytes":14680064,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":52,"num_read_kb":15272,"num_write":108,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.b","version":"96'22","reported_seq":391,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.580014+0000","last_change":"2024-09-17T17:30:51.542828+0000","last_active":"2024-09-17T17:30:57.580014+0000","last_peered":"2024-09-17T17:30:57.580014+0000","last_clean":"2024-09-17T17:30:57.580014+0000","last_became_active":"2024-09-17T17:30:47.358299+0000","last_became_peered":"2024-09-17T17:30:47.358299+0000","last_unstale":"2024-09-17T17:30:57.580014+0000","last_undegraded":"2024-09-17T17:30:57.580014+0000","last_fullsized":"2024-09-17T17:30:57.580014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:12.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001376436,"stat_sum":{"num_bytes":2310144,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":6719,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4538368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,10,6,8,7,14],"acting":[13,3,10,6,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.c","version":"100'174","reported_seq":509,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582700+0000","last_change":"2024-09-17T17:30:51.567084+0000","last_active":"2024-09-17T17:30:57.582700+0000","last_peered":"2024-09-17T17:30:57.582700+0000","last_clean":"2024-09-17T17:30:57.582700+0000","last_became_active":"2024-09-17T17:29:44.895885+0000","last_became_peered":"2024-09-17T17:29:44.895885+0000","last_unstale":"2024-09-17T17:30:57.582700+0000","last_undegraded":"2024-09-17T17:30:57.582700+0000","last_fullsized":"2024-09-17T17:30:57.582700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:01.541591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026776215999999999,"stat_sum":{"num_bytes":17776640,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":50,"num_read_kb":11988,"num_write":102,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,12,4,1,9],"acting":[11,13,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.d","version":"100'110","reported_seq":445,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-17T17:31:01.612532+0000","last_change":"2024-09-17T17:30:51.569527+0000","last_active":"2024-09-17T17:31:01.612532+0000","last_peered":"2024-09-17T17:31:01.612532+0000","last_clean":"2024-09-17T17:31:01.612532+0000","last_became_active":"2024-09-17T17:29:44.896111+0000","last_became_peered":"2024-09-17T17:29:44.896111+0000","last_unstale":"2024-09-17T17:31:01.612532+0000","last_undegraded":"2024-09-17T17:31:01.612532+0000","last_fullsized":"2024-09-17T17:31:01.612532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:11.750198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.025471486000000002,"stat_sum":{"num_bytes":9322496,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":19,"num_read_kb":9229,"num_write":58,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,5],"acting":[7,3,0,12,9,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"}]},{"pgid":"3.e","version":"100'66","reported_seq":403,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T17:30:58.590014+0000","last_change":"2024-09-17T17:30:52.137405+0000","last_active":"2024-09-17T17:30:58.590014+0000","last_peered":"2024-09-17T17:30:58.590014+0000","last_clean":"2024-09-17T17:30:58.590014+0000","last_became_active":"2024-09-17T17:29:44.895935+0000","last_became_peered":"2024-09-17T17:29:44.895935+0000","last_unstale":"2024-09-17T17:30:58.590014+0000","last_undegraded":"2024-09-17T17:30:58.590014+0000","last_fullsized":"2024-09-17T17:30:58.590014+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:01.710224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.590164252,"stat_sum":{"num_bytes":6520832,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":18,"num_read_kb":3129,"num_write":52,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,5],"acting":[8,14,6,11,13,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"unknown","last_fresh":"2024-09-17T17:30:57.580447+0000","last_change":"2024-09-17T17:30:58.563434+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T17:30:57.580447+0000","last_undegraded":"2024-09-17T17:30:57.580447+0000","last_fullsized":"2024-09-17T17:30:57.580447+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:29:43.627844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,5,10,0,9,6],"acting":[13,14,5,10,0,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.10","version":"77'53","reported_seq":189,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.584098+0000","last_change":"2024-09-17T17:30:51.556879+0000","last_active":"2024-09-17T17:30:57.584098+0000","last_peered":"2024-09-17T17:30:57.584098+0000","last_clean":"2024-09-17T17:30:57.584098+0000","last_became_active":"2024-09-17T17:30:47.354125+0000","last_became_peered":"2024-09-17T17:30:47.354125+0000","last_unstale":"2024-09-17T17:30:57.584098+0000","last_undegraded":"2024-09-17T17:30:57.584098+0000","last_fullsized":"2024-09-17T17:30:57.584098+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'52","last_scrub_stamp":"2024-09-17T17:30:26.907313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:26.907313+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:50.061590+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.011082936,"stat_sum":{"num_bytes":2342912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7,14,5,9,11],"acting":[6,0,7,14,5,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.11","version":"60'25","reported_seq":131,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T17:30:58.590307+0000","last_change":"2024-09-17T17:30:51.540542+0000","last_active":"2024-09-17T17:30:58.590307+0000","last_peered":"2024-09-17T17:30:58.590307+0000","last_clean":"2024-09-17T17:30:58.590307+0000","last_became_active":"2024-09-17T17:30:36.353414+0000","last_became_peered":"2024-09-17T17:30:36.353414+0000","last_unstale":"2024-09-17T17:30:58.590307+0000","last_undegraded":"2024-09-17T17:30:58.590307+0000","last_fullsized":"2024-09-17T17:30:58.590307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'25","last_scrub_stamp":"2024-09-17T17:30:37.309789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:37.309789+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:40.299384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079811599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,15,10,5,14,2],"acting":[4,11,15,10,5,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.12","version":"100'95","reported_seq":510,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.578649+0000","last_change":"2024-09-17T17:30:51.564660+0000","last_active":"2024-09-17T17:30:57.578649+0000","last_peered":"2024-09-17T17:30:57.578649+0000","last_clean":"2024-09-17T17:30:57.578649+0000","last_became_active":"2024-09-17T17:30:39.350802+0000","last_became_peered":"2024-09-17T17:30:39.350802+0000","last_unstale":"2024-09-17T17:30:57.578649+0000","last_undegraded":"2024-09-17T17:30:57.578649+0000","last_fullsized":"2024-09-17T17:30:57.578649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'48","last_scrub_stamp":"2024-09-17T17:30:24.093396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:24.093396+0000","objects_scrubbed":3,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:35.435680+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0.018394490999999999,"stat_sum":{"num_bytes":3489792,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":3,"num_read_kb":1,"num_write":11,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6160384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,7,5,4,11,3,15],"acting":[14,7,5,4,11,3,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"2"}]},{"pgid":"3.13","version":"100'68","reported_seq":500,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T17:30:57.582563+0000","last_change":"2024-09-17T17:30:51.552659+0000","last_active":"2024-09-17T17:30:57.582563+0000","last_peered":"2024-09-17T17:30:57.582563+0000","last_clean":"2024-09-17T17:30:57.582563+0000","last_became_active":"2024-09-17T17:30:46.350583+0000","last_became_peered":"2024-09-17T17:30:46.350583+0000","last_unstale":"2024-09-17T17:30:57.582563+0000","last_undegraded":"2024-09-17T17:30:57.582563+0000","last_fullsized":"2024-09-17T17:30:57.582563+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'52","last_scrub_stamp":"2024-09-17T17:30:27.908152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:43.627844+0000","last_clean_scrub_stamp":"2024-09-17T17:30:27.908152+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:41.948407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.012392781,"stat_sum":{"num_bytes":4063232,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7634944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,6,5,4,13],"acting":[11,7,2,6,5,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":122404864,"num_objects":109,"num_object_clones":61,"num_object_copies":763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":17,"num_read":220,"num_read_kb":77779,"num_write":434,"num_write_kb":84832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":42123264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":255221760,"data_stored":372764672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1740,"ondisk_log_size":1740,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247829,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48092,"kb_used_data":12740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323748,"statfs":{"total":96636764160,"available":96587517952,"internally_reserved":0,"allocated":13045760,"data_stored":21319007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247829,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50200,"kb_used_data":19008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321640,"statfs":{"total":96636764160,"available":96585359360,"internally_reserved":0,"allocated":19464192,"data_stored":28866033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247830,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45964,"kb_used_data":14776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325876,"statfs":{"total":96636764160,"available":96589697024,"internally_reserved":0,"allocated":15130624,"data_stored":14945631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,5,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313238,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71440,"kb_used_data":18040,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94300400,"statfs":{"total":96636764160,"available":96563609600,"internally_reserved":0,"allocated":18472960,"data_stored":26463241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46536,"kb_used_data":15344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325304,"statfs":{"total":96636764160,"available":96589111296,"internally_reserved":0,"allocated":15712256,"data_stored":24475121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313239,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48616,"kb_used_data":17420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323224,"statfs":{"total":96636764160,"available":96586981376,"internally_reserved":0,"allocated":17838080,"data_stored":23769898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280534,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52912,"kb_used_data":21720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318928,"statfs":{"total":96636764160,"available":96582582272,"internally_reserved":0,"allocated":22241280,"data_stored":26908689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280533,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45972,"kb_used_data":14776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325868,"statfs":{"total":96636764160,"available":96589688832,"internally_reserved":0,"allocated":15130624,"data_stored":21935601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280534,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50948,"kb_used_data":15600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94320892,"statfs":{"total":96636764160,"available":96584593408,"internally_reserved":0,"allocated":15974400,"data_stored":20485617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280534,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49028,"kb_used_data":13680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322812,"statfs":{"total":96636764160,"available":96586559488,"internally_reserved":0,"allocated":14008320,"data_stored":22656516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,7,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280533,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44724,"kb_used_data":13532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327116,"statfs":{"total":96636764160,"available":96590966784,"internally_reserved":0,"allocated":13856768,"data_stored":19875313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280533,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47124,"kb_used_data":15932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324716,"statfs":{"total":96636764160,"available":96588509184,"internally_reserved":0,"allocated":16314368,"data_stored":21435889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280533,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48972,"kb_used_data":21936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322868,"statfs":{"total":96636764160,"available":96586616832,"internally_reserved":0,"allocated":22462464,"data_stored":34326001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280534,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55032,"kb_used_data":23840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316808,"statfs":{"total":96636764160,"available":96580411392,"internally_reserved":0,"allocated":24412160,"data_stored":29267460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247830,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71036,"kb_used_data":17636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94300804,"statfs":{"total":96636764160,"available":96564023296,"internally_reserved":0,"allocated":18059264,"data_stored":28343647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247829,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50300,"kb_used_data":19108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321540,"statfs":{"total":96636764160,"available":96585256960,"internally_reserved":0,"allocated":19566592,"data_stored":23685137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20131840,"data_stored":25452544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16846848,"data_stored":25505792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20885504,"data_stored":33398784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22831104,"data_stored":28340224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16465920,"data_stored":27406336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14397440,"data_stored":19558400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17457152,"data_stored":22228992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16195584,"data_stored":22802432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":13537280,"data_stored":14008320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":12427264,"data_stored":21729280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":12279808,"data_stored":18948096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":17887232,"data_stored":27938816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":11452416,"data_stored":20381696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":13553664,"data_stored":21008384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":14135296,"data_stored":23547904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":14737408,"data_stored":20508672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:31:03.050 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-17T17:31:03.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-17T17:31:03.051 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:31:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:31:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:31:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 37/35 ECSubWriteReply(tid=17, last_complete=37'13, committed=1, applied=1))", 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 74.047681245000007, 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.068701408000000005, 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000937+0000", 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6449999999999998e-06 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068896+0000", 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.067959031000000003 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068909+0000", 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3416e-05 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069008+0000", 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8690999999999994e-05 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069634+0000", 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062589099999999995 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 72.364749986999996, 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079875469999999994, 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2e-07 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684336+0000", 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047134800000000001 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684439+0000", 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010291800000000001 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684449+0000", 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.027e-05 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684596+0000", 2024-09-17T17:31:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000146749 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684607+0000", 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1421e-05 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691787+0000", 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071804069999999998 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691851+0000", 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3913999999999998e-05 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 72.363259580999994, 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089994210000000005, 2024-09-17T17:31:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4100000000000001e-07 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685373+0000", 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9063999999999999e-05 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687383+0000", 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002009355 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687391+0000", 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9699999999999999e-06 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687457+0000", 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5834999999999996e-05 2024-09-17T17:31:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9990000000000003e-06 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694321+0000", 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068546010000000001 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694354+0000", 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2456000000000002e-05 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:191 3.5s0 3:a4d61510:::smithi10238019-32:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 72.361176718999999, 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.008961732, 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 191 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.068 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687439+0000", 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1799999999999999e-06 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687504+0000", 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4770000000000005e-05 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687516+0000", 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2007e-05 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687531+0000", 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4803e-05 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687595+0000", 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3697999999999999e-05 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687756+0000", 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000161846 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.690070+0000", 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0023137639999999998 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695782+0000", 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057116959999999996 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695908+0000", 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000126339 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.696399+0000", 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049062900000000002 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:407 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 60.710536570000002, 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011701008000000001, 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 407 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338080+0000", 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2819999999999999e-06 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6289999999999999e-06 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338108+0000", 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9514e-05 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338117+0000", 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.132e-06 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1446999999999995e-05 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338196+0000", 2024-09-17T17:31:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6350000000000006e-06 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341655+0000", 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034593860000000001 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342199+0000", 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054402400000000005 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349778+0000", 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075789589999999997 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:408 3.5s0 3:a96b502c:::smithi10238019-42:head [read 53220~651292,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 60.710374643000002, 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012064669, 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 408 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338240+0000", 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338257+0000", 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6643e-05 2024-09-17T17:31:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338270+0000", 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3081000000000001e-05 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338277+0000", 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3780000000000003e-06 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338331+0000", 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4286999999999997e-05 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338338+0000", 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3609999999999997e-06 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342681+0000", 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043432749999999997 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342775+0000", 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4392e-05 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.350304+0000", 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007528435 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:409 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 60.710261887999998, 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012665852, 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 409 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338353+0000", 2024-09-17T17:31:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.09e-07 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338358+0000", 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7859999999999996e-06 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338366+0000", 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6049999999999998e-06 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338372+0000", 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1739999999999997e-06 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338439+0000", 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7039999999999995e-05 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338446+0000", 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1160000000000001e-06 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343114+0000", 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0046672179999999999 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343207+0000", 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3170000000000004e-05 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351018+0000", 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078109340000000003 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 60.696107507999997, 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.988933536, 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352507+0000", 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1500000000000003e-07 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352532+0000", 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4485999999999999e-05 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340513+0000", 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.98798163100000003 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340522+0000", 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9379999999999998e-06 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340625+0000", 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010273 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340633+0000", 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:31:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341398+0000", 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00076522600000000004 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341440+0000", 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1548999999999998e-05 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/35 ECSubWrite(tid=50, reqid=client.4566.0:441, at_version=0'0, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 59.185049988000003, 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.121580543, 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863565+0000", 2024-09-17T17:31:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6729999999999999e-06 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863588+0000", 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2756e-05 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984520+0000", 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.120931489 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984547+0000", 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6877999999999999e-05 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984631+0000", 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4246000000000006e-05 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984638+0000", 2024-09-17T17:31:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3509999999999996e-06 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985117+0000", 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047851700000000003 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985144+0000", 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7633000000000001e-05 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.547959454999997, 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.090576745, 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500656+0000", 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0430000000000002e-06 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500772+0000", 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000115338 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590079+0000", 2024-09-17T17:31:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089307485000000006 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590103+0000", 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4131e-05 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590159+0000", 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5417000000000002e-05 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590177+0000", 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7819e-05 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591201+0000", 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023939 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591231+0000", 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0573e-05 2024-09-17T17:31:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 75/74 ECSubWrite(tid=94, reqid=osd.10.0:94, at_version=75'58, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 36.696491829000003, 2024-09-17T17:31:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086672569999999994, 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352123+0000", 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352143+0000", 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0440000000000001e-05 2024-09-17T17:31:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352170+0000", 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6503e-05 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352177+0000", 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6429999999999995e-06 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352236+0000", 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8711000000000001e-05 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352245+0000", 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360755+0000", 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085095469999999993 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360789+0000", 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4678999999999998e-05 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 78/74 ECSubWrite(tid=102, reqid=client.4566.0:771, at_version=78'64, trim_to=0'0, roll_forward_to=78'60))", 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 33.666760488000001, 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96796263999999999, 2024-09-17T17:31:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381854+0000", 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4e-07 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381870+0000", 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6076e-05 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348683+0000", 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96681319499999996 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348717+0000", 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348821+0000", 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010401800000000001 2024-09-17T17:31:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348833+0000", 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2172e-05 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349780+0000", 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00094660899999999995 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349816+0000", 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6025999999999997e-05 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 79/74 ECSubWrite(tid=108, reqid=client.4566.0:788, at_version=79'60, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.683054435999999, 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010065843999999999, 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:31:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365560+0000", 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5199999999999995e-07 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365578+0000", 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8145999999999999e-05 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374667+0000", 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.009088245 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374697+0000", 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0402000000000001e-05 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374763+0000", 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5912000000000002e-05 2024-09-17T17:31:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374776+0000", 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2497000000000001e-05 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375591+0000", 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00081505399999999995 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375625+0000", 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4635999999999998e-05 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 80/74 ECSubWrite(tid=112, reqid=osd.10.0:111, at_version=0'0, trim_to=0'0, roll_forward_to=80'62))", 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 31.696230605, 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076437279999999998, 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352384+0000", 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0249999999999999e-06 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352400+0000", 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5628999999999999e-05 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352437+0000", 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6813000000000001e-05 2024-09-17T17:31:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352448+0000", 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0882e-05 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352672+0000", 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00022395999999999999 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352683+0000", 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1656e-05 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.359996+0000", 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073125120000000002 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.360027+0000", 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1251000000000003e-05 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.13.22:9 3.5s0 3:a4d61510:::smithi10238019-32:head [assert-version v27,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.706523194999999, 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015638470000000002, 2024-09-17T17:31:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342092+0000", 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7e-06 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342098+0000", 2024-09-17T17:31:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8690000000000001e-06 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342114+0000", 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5954999999999999e-05 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342122+0000", 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3919999999999997e-06 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342179+0000", 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7768e-05 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342187+0000", 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3819999999999998e-06 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348515+0000", 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063278780000000003 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348672+0000", 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015732800000000001 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.357729+0000", 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090571980000000007 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 85/76 ECSubWrite(tid=194, reqid=osd.12.0:193, at_version=0'0, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:31:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.696310758999999, 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0091352759999999995, 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352304+0000", 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4e-07 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352316+0000", 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1864e-05 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352360+0000", 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4478999999999999e-05 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352370+0000", 2024-09-17T17:31:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5829999999999996e-06 2024-09-17T17:31:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352437+0000", 2024-09-17T17:31:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7521000000000006e-05 2024-09-17T17:31:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352447+0000", 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2180000000000005e-06 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361372+0000", 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089248839999999992 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361438+0000", 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6786999999999996e-05 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 85/74 ECSubWrite(tid=125, reqid=osd.10.0:125, at_version=85'76, trim_to=0'0, roll_forward_to=83'74))", 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.695676424999998, 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085212919999999998, 2024-09-17T17:31:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:31:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352938+0000", 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.04e-07 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352958+0000", 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0486e-05 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353044+0000", 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5852000000000004e-05 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353050+0000", 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2280000000000003e-06 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353110+0000", 2024-09-17T17:31:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9478999999999998e-05 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353121+0000", 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1426e-05 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361426+0000", 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0083047659999999999 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361459+0000", 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2851000000000001e-05 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:922 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 15.703100682000001, 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010998030000000001, 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 922 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345515+0000", 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4219999999999999e-06 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345520+0000", 2024-09-17T17:31:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2909999999999998e-06 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345543+0000", 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3323e-05 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345551+0000", 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3549999999999999e-06 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345653+0000", 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010261500000000001 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345661+0000", 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1310000000000006e-06 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.352634+0000", 2024-09-17T17:31:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0069729299999999996 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.352757+0000", 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000122714 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.356511+0000", 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0037542489999999999 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:923 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 15.703046306999999, 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011747277, 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 923 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345568+0000", 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6300000000000005e-07 2024-09-17T17:31:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345574+0000", 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5219999999999998e-06 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345580+0000", 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1120000000000001e-06 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345585+0000", 2024-09-17T17:31:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3229999999999997e-06 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345814+0000", 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000228989 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345820+0000", 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2419999999999997e-06 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.353324+0000", 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075038520000000001 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.353431+0000", 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107129 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.357315+0000", 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003883545 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:924 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 15.702885714000001, 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012050959, 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 924 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345729+0000", 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.015e-06 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345734+0000", 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2909999999999998e-06 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345745+0000", 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0134e-05 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345760+0000", 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5775999999999999e-05 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345844+0000", 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3635999999999997e-05 2024-09-17T17:31:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345849+0000", 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2100000000000001e-06 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.354091+0000", 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0082413780000000006 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.354183+0000", 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2315e-05 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.357779+0000", 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003596204 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:03.159 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:03.161 INFO:tasks.ceph.osd.9.smithi026.stderr:2024-09-17T17:31:03.159+0000 7f1c87221640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33233) UID: 0 2024-09-17T17:31:03.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:3.fs1 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:3.es0 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:3.bs4 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:3.as4 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:3.7s1 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:3.17s1 2024-09-17T17:31:03.213 INFO:teuthology.orchestra.run.smithi007.stdout:2.5 2024-09-17T17:31:03.262 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:31:03.261+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:31:03.274 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:03.274 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:03.274 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:31:03.274 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:03.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T17:31:03.362 INFO:tasks.ceph.osd.10.smithi027.stderr:2024-09-17T17:31:03.361+0000 7fb9ac333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33946) UID: 0 2024-09-17T17:31:03.430 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:31:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:31:03.431 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:03.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T17:31:03.464 INFO:tasks.ceph.osd.5.smithi026.stderr:2024-09-17T17:31:03.462+0000 7f9fd6231640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33228) UID: 0 2024-09-17T17:31:03.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-8 --journal-path /var/lib/ceph/osd/ceph-8/journal --op export --pgid 3.bs4 --file /var/log/ceph/exp.3.bs4.8 2024-09-17T17:31:03.550 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:03.550 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:31:03.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:31:03.550 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:31:03.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 42/35 ECSubWrite(tid=80, reqid=client.4566.0:274, at_version=42'13, trim_to=0'0, roll_forward_to=40'11))", 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.839732583, 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003864363, 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707918+0000", 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.707919+0000", 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708479+0000", 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055997299999999996 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708513+0000", 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4359000000000001e-05 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708524+0000", 2024-09-17T17:31:03.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0346999999999999e-05 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708600+0000", 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999997e-05 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.708619+0000", 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8490000000000001e-05 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711752+0000", 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031327600000000001 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:53.711783+0000", 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1035999999999999e-05 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.555 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 43/35 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=43'17, trim_to=0'0, roll_forward_to=42'15))", 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.850084283000001, 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010715961, 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697567+0000", 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697586+0000", 2024-09-17T17:31:03.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8556999999999999e-05 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697629+0000", 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2982999999999998e-05 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697637+0000", 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697698+0000", 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0534e-05 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.697710+0000", 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1745e-05 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708248+0000", 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010538476999999999 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708283+0000", 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4431999999999998e-05 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=39, reqid=osd.8.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=43'29))", 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.846800157999994, 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074300950000000003, 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700851+0000", 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700852+0000", 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1969999999999999e-06 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700867+0000", 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4960000000000001e-05 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700917+0000", 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0285999999999999e-05 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.700931+0000", 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3332999999999999e-05 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701036+0000", 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105467 2024-09-17T17:31:03.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701054+0000", 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7827999999999999e-05 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708242+0000", 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071881929999999998 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708281+0000", 2024-09-17T17:31:03.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8831000000000001e-05 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 43/32 ECSubWrite(tid=20, reqid=osd.7.0:19, at_version=0'0, trim_to=0'0, roll_forward_to=43'16))", 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.846216616999996, 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068976910000000001, 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.565 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701434+0000", 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701435+0000", 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701447+0000", 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2424e-05 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1851e-05 2024-09-17T17:31:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701527+0000", 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7069999999999992e-06 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701592+0000", 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4825000000000004e-05 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.701600+0000", 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5420000000000005e-06 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708305+0000", 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067049680000000004 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.708332+0000", 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6899000000000001e-05 2024-09-17T17:31:03.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 48/32 ECSubWrite(tid=41, reqid=osd.8.0:40, at_version=0'0, trim_to=0'0, roll_forward_to=48'31))", 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.827426969000001, 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77214092400000001, 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720224+0000", 2024-09-17T17:31:03.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:59.720235+0000", 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0353999999999999e-05 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491642+0000", 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.77140694700000001 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491651+0000", 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491754+0000", 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010309 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.491768+0000", 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144999999999999e-05 2024-09-17T17:31:03.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492327+0000", 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000558998 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:00.492365+0000", 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969000000000002e-05 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.574 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.0s3 51/35 ECSubWrite(tid=60, reqid=osd.12.0:60, at_version=51'16, trim_to=0'0, roll_forward_to=50'14))", 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.211644376999999, 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028553210000000001, 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336007+0000", 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.336043+0000", 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6211000000000003e-05 2024-09-17T17:31:03.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337026+0000", 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098293199999999995 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337033+0000", 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4919999999999996e-06 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337104+0000", 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0616000000000003e-05 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.337116+0000", 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1454000000000001e-05 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338825+0000", 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001709824 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338862+0000", 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6477000000000003e-05 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 57/32 ECSubWrite(tid=49, reqid=osd.8.0:49, at_version=57'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.170763018999999, 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026169420000000001, 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376888+0000", 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2600000000000001e-07 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376909+0000", 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0905999999999999e-05 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376963+0000", 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.393e-05 2024-09-17T17:31:03.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376970+0000", 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1130000000000003e-06 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377048+0000", 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8391000000000002e-05 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377057+0000", 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379423+0000", 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023655939999999999 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379505+0000", 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2259e-05 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/35 ECSubWrite(tid=148, reqid=client.4566.0:490, at_version=57'22, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.169307527000001, 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028134560000000002, 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378343+0000", 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378344+0000", 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.47e-07 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378373+0000", 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.917e-05 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378410+0000", 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6259e-05 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378418+0000", 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5299999999999996e-06 2024-09-17T17:31:03.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378488+0000", 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9487999999999996e-05 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378497+0000", 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3889999999999994e-06 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381121+0000", 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002624103 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.381157+0000", 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.557e-05 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 79/74 ECSubWrite(tid=247, reqid=client.4566.0:796, at_version=79'135, trim_to=0'0, roll_forward_to=78'133))", 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:31:03.588 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.172712191999999, 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027697429999999999, 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:31:03.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.374939+0000", 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376108+0000", 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011690730000000001 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376132+0000", 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3760999999999998e-05 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376140+0000", 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.191e-06 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376198+0000", 2024-09-17T17:31:03.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7695000000000002e-05 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.376205+0000", 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2089999999999999e-06 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.377683+0000", 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014781639999999999 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:30.377708+0000", 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5445000000000001e-05 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=272, reqid=client.4566.0:841, at_version=85'148, trim_to=0'0, roll_forward_to=83'146))", 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.184158125, 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028713852000000002, 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.363493+0000", 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-07 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.364388+0000", 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089506500000000001 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390674+0000", 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026285665999999999 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390684+0000", 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0424e-05 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390748+0000", 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4132000000000002e-05 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390756+0000", 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1249999999999993e-06 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.392176+0000", 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001419783 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.392207+0000", 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0375e-05 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=273, reqid=client.4566.0:843, at_version=85'149, trim_to=0'0, roll_forward_to=83'146))", 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.390583+0000", 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.15706746, 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048863159999999999, 2024-09-17T17:31:03.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390583+0000", 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390583+0000", 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390585+0000", 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.246e-06 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.390655+0000", 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9933999999999998e-05 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394610+0000", 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039550799999999997 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394619+0000", 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2699999999999993e-06 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394677+0000", 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7744000000000001e-05 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394688+0000", 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1362999999999999e-05 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.395431+0000", 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074258799999999997 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.395470+0000", 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9091000000000002e-05 2024-09-17T17:31:03.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=274, reqid=client.4566.0:844, at_version=85'150, trim_to=0'0, roll_forward_to=85'148))", 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.394381+0000", 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.153269472000002, 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044568630000000001, 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394381+0000", 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394381+0000", 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394382+0000", 2024-09-17T17:31:03.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5300000000000003e-07 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.394589+0000", 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020663800000000001 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397623+0000", 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030343689999999999 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397629+0000", 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397681+0000", 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1746999999999998e-05 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397692+0000", 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1686e-05 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.398799+0000", 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001106671 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.398838+0000", 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9362e-05 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=275, reqid=client.4566.0:845, at_version=85'151, trim_to=0'0, roll_forward_to=85'149))", 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.397416+0000", 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.15023459, 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052212270000000002, 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397416+0000", 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397416+0000", 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397417+0000", 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T17:31:03.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.397609+0000", 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000192146 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401693+0000", 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040836860000000004 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401702+0000", 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1160000000000005e-06 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401760+0000", 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7853999999999999e-05 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.401771+0000", 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0988999999999999e-05 2024-09-17T17:31:03.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402611+0000", 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00084015999999999995 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402638+0000", 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6565999999999999e-05 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.16s6 85/74 ECSubWrite(tid=277, reqid=client.4566.0:847, at_version=85'153, trim_to=0'0, roll_forward_to=85'150))", 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:36.402653+0000", 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.144997920000002, 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11847311000000001, 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402653+0000", 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402653+0000", 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402654+0000", 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.046e-06 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.402670+0000", 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.641e-05 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520286+0000", 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.117615488 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520302+0000", 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5724000000000001e-05 2024-09-17T17:31:03.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520454+0000", 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015243400000000001 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.520474+0000", 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9877e-05 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.521094+0000", 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062014800000000005 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:36.521126+0000", 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1983000000000002e-05 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:31:03.617 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.204313566, 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031028610000000002, 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343337+0000", 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.48e-07 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343358+0000", 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0500999999999999e-05 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343402+0000", 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4209000000000003e-05 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343415+0000", 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3072e-05 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343484+0000", 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8427000000000001e-05 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.343497+0000", 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3077e-05 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.346406+0000", 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029091080000000001 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.346440+0000", 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4319000000000001e-05 2024-09-17T17:31:03.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.es5 91/32 ECSubWrite(tid=75, reqid=client.4566.0:895, at_version=91'58, trim_to=0'0, roll_forward_to=83'57))", 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.383051+0000", 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.164600324999999, 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00300592, 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383051+0000", 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383051+0000", 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383051+0000", 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-17T17:31:03.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383450+0000", 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000398642 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383479+0000", 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9133e-05 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383490+0000", 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253000000000001e-05 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383574+0000", 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4134000000000005e-05 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.383589+0000", 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4559000000000001e-05 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.386020+0000", 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002431169 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.386057+0000", 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6541999999999997e-05 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.ds2 96/32 ECSubWrite(tid=112, reqid=osd.7.0:112, at_version=96'97, trim_to=0'0, roll_forward_to=93'95))", 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:47.346398+0000", 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.201252564000001, 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033125849999999998, 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346398+0000", 2024-09-17T17:31:03.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346398+0000", 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346399+0000", 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346430+0000", 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1653999999999999e-05 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346482+0000", 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.164e-05 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346511+0000", 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8785999999999998e-05 2024-09-17T17:31:03.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346576+0000", 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5446999999999999e-05 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.346588+0000", 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1934e-05 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.349667+0000", 2024-09-17T17:31:03.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00307932 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.349711+0000", 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3585999999999999e-05 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 96/74 ECSubWrite(tid=328, reqid=osd.13.0:328, at_version=96'42, trim_to=0'0, roll_forward_to=93'40))", 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:47.347342+0000", 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.200308687, 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029719210000000002, 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347342+0000", 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347342+0000", 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347342+0000", 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6400000000000001e-07 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347367+0000", 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4978e-05 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347412+0000", 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4628e-05 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347421+0000", 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2930000000000006e-06 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347488+0000", 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6853000000000005e-05 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.347499+0000", 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0814e-05 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.350271+0000", 2024-09-17T17:31:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002772142 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.350314+0000", 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3049000000000001e-05 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s5 96/74 ECSubWrite(tid=330, reqid=client.4566.0:927, at_version=96'46, trim_to=0'0, roll_forward_to=96'42))", 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:47.360081+0000", 2024-09-17T17:31:03.637 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.187569532000001, 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030243219999999999, 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360081+0000", 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360081+0000", 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360082+0000", 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1900000000000001e-07 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360286+0000", 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020378700000000001 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360326+0000", 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0104000000000001e-05 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360335+0000", 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0480000000000004e-06 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360403+0000", 2024-09-17T17:31:03.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8188999999999994e-05 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.360414+0000", 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0223e-05 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.363069+0000", 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026551270000000002 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.363106+0000", 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7024999999999999e-05 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.18s2 100/75 ECSubWrite(tid=176, reqid=client.4566.0:964, at_version=100'31, trim_to=0'0, roll_forward_to=76'29))", 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:51.787641+0000", 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.760009626, 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063470449999999999, 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787641+0000", 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787641+0000", 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787642+0000", 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791790+0000", 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004148583 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791817+0000", 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6829999999999999e-05 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791823+0000", 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4430000000000002e-06 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791864+0000", 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1736000000000002e-05 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791871+0000", 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1589999999999998e-06 2024-09-17T17:31:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.793947+0000", 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020764360000000001 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.793988+0000", 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1257999999999997e-05 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:03.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:03.647 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:03.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T17:31:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'unique_pool_0' removed 2024-09-17T17:31:03.650 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T17:31:03.662 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T17:31:03.663 INFO:tasks.ceph.osd.6.smithi027.stderr:2024-09-17T17:31:03.646+0000 7f3e17f70640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33941) UID: 0 2024-09-17T17:31:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:31:03.805 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:03.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T17:31:03.890 INFO:teuthology.orchestra.run.smithi007.stderr:Exporting 3.bs4 info 3.bs4( v 96'22 (0'0,96'22] local-lis/les=95/96 n=1 ec=32/32 lis/c=95/95 les/c/f=96/96/0 sis=95) 2024-09-17T17:31:03.891 INFO:teuthology.orchestra.run.smithi007.stderr:Read 4#3:d4565dea:::smithi10238019-24:head# 2024-09-17T17:31:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:Export successful 2024-09-17T17:31:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:03.923 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:03.923 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:31:03.923 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:31:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:03.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 51/35 ECSubWrite(tid=94, reqid=client.4566.0:406, at_version=51'30, trim_to=0'0, roll_forward_to=50'28))", 2024-09-17T17:31:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.695370685, 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003277556, 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343211+0000", 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343212+0000", 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343225+0000", 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3414999999999999e-05 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343249+0000", 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3983999999999999e-05 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343254+0000", 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5249999999999996e-06 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343307+0000", 2024-09-17T17:31:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2167999999999999e-05 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.343315+0000", 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8580000000000006e-06 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346453+0000", 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003137908 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.346488+0000", 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5021999999999998e-05 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 56/35 ECSubWrite(tid=57, reqid=osd.10.0:57, at_version=56'41, trim_to=0'0, roll_forward_to=53'39))", 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.703318109999998, 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036979209999999998, 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335263+0000", 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335264+0000", 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335298+0000", 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3837000000000001e-05 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335654+0000", 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035630699999999999 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335665+0000", 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1620999999999999e-05 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335751+0000", 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5692999999999994e-05 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.335763+0000", 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140000000000001e-05 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338926+0000", 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031629399999999999 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338961+0000", 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5129999999999997e-05 2024-09-17T17:31:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.2s3 56/32 ECSubWrite(tid=37, reqid=osd.14.0:37, at_version=56'34, trim_to=0'0, roll_forward_to=53'32))", 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.700512396000001, 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003185701, 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338069+0000", 2024-09-17T17:31:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338094+0000", 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4428000000000001e-05 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011644700000000001 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338219+0000", 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9020000000000005e-06 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338416+0000", 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196558 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338424+0000", 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.585e-06 2024-09-17T17:31:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341209+0000", 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027847050000000002 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341255+0000", 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.700471166, 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004124801, 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:31:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338110+0000", 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338111+0000", 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9600000000000001e-07 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338129+0000", 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8003e-05 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338243+0000", 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114367 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338249+0000", 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3160000000000002e-06 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338788+0000", 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053876399999999995 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338797+0000", 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.85e-06 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342196+0000", 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033993830000000002 2024-09-17T17:31:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342235+0000", 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8921999999999998e-05 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 56/35 ECSubWrite(tid=138, reqid=osd.6.0:138, at_version=56'13, trim_to=0'0, roll_forward_to=53'11))", 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.700372231999999, 2024-09-17T17:31:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040538249999999996, 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338209+0000", 2024-09-17T17:31:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338210+0000", 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.99e-07 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338233+0000", 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3623999999999999e-05 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338275+0000", 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2249000000000002e-05 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338284+0000", 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2609999999999993e-06 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338436+0000", 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152032 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338448+0000", 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2469999999999999e-05 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342230+0000", 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037816899999999999 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.342263+0000", 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3300000000000003e-05 2024-09-17T17:31:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 56/35 ECSubWrite(tid=125, reqid=osd.11.0:125, at_version=56'44, trim_to=0'0, roll_forward_to=53'42))", 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.700311620999997, 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029941270000000001, 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338270+0000", 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T17:31:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338289+0000", 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8340000000000001e-05 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338322+0000", 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3419e-05 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338332+0000", 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0086e-05 2024-09-17T17:31:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338873+0000", 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054120000000000004 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.338882+0000", 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1329999999999999e-06 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341233+0000", 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023510699999999998 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.341264+0000", 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1365999999999998e-05 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:31:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.666206789, 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285746, 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372375+0000", 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372399+0000", 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4416000000000001e-05 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372479+0000", 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9556999999999997e-05 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372487+0000", 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7589999999999997e-06 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372561+0000", 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3807000000000007e-05 2024-09-17T17:31:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.372571+0000", 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0532e-05 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375622+0000", 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030507860000000002 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.375661+0000", 2024-09-17T17:31:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8634000000000003e-05 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=137, reqid=osd.11.0:137, at_version=57'50, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.665477023000001, 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004179131, 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373105+0000", 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373125+0000", 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9626000000000001e-05 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373159+0000", 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4190999999999998e-05 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373168+0000", 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6999999999999997e-06 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373237+0000", 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9306000000000004e-05 2024-09-17T17:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373246+0000", 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377245+0000", 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039987390000000003 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377284+0000", 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8423999999999997e-05 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/35 ECSubWrite(tid=138, reqid=osd.11.0:138, at_version=57'52, trim_to=0'0, roll_forward_to=56'48))", 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.665130904999998, 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048101309999999996, 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373451+0000", 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1800000000000005e-07 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373476+0000", 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4712999999999999e-05 2024-09-17T17:31:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373535+0000", 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8536999999999999e-05 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373545+0000", 2024-09-17T17:31:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0142e-05 2024-09-17T17:31:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374163+0000", 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061771899999999997 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.374172+0000", 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.081e-06 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378233+0000", 2024-09-17T17:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040611509999999998 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.378261+0000", 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000002e-05 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:31:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.66508082, 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038195389999999998, 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373501+0000", 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373518+0000", 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373595+0000", 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6206999999999997e-05 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373601+0000", 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6039999999999998e-06 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373659+0000", 2024-09-17T17:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7442000000000003e-05 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.373668+0000", 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2040000000000003e-06 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377300+0000", 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00363182 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.377320+0000", 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0598000000000002e-05 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/35 ECSubWrite(tid=147, reqid=client.4566.0:489, at_version=57'21, trim_to=0'0, roll_forward_to=48'19))", 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.662479294999997, 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034371530000000001, 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376102+0000", 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376103+0000", 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5899999999999998e-07 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376128+0000", 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5109e-05 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376199+0000", 2024-09-17T17:31:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1783e-05 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376209+0000", 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1710000000000008e-06 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376272+0000", 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749000000000002e-05 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.376281+0000", 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8899999999999996e-06 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379498+0000", 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003216605 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:08.379539+0000", 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1587e-05 2024-09-17T17:31:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 68/67 ECSubWrite(tid=75, reqid=client.4566.0:644, at_version=68'52, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.338634716000001, 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52365443700000003, 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699947+0000", 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699948+0000", 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999998e-07 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.699964+0000", 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6784000000000001e-05 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222607+0000", 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52264301099999999 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222621+0000", 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3923e-05 2024-09-17T17:31:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222745+0000", 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012366800000000001 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.222753+0000", 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6490000000000008e-06 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223566+0000", 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081285500000000002 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.223601+0000", 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5744000000000002e-05 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/74 ECSubWrite(tid=163, reqid=osd.6.0:163, at_version=75'29, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:31:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.687519743999999, 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.34608951199999999, 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351062+0000", 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351063+0000", 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9999999999999997e-07 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.351085+0000", 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2129000000000001e-05 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696028+0000", 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.344942945 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696043+0000", 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5353e-05 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696175+0000", 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131996 2024-09-17T17:31:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.696189+0000", 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3573e-05 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697107+0000", 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091841899999999996 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.697151+0000", 2024-09-17T17:31:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4397000000000002e-05 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.13s4 80/74 ECSubWrite(tid=111, reqid=osd.10.0:111, at_version=80'62, trim_to=0'0, roll_forward_to=79'60))", 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.691161170999997, 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003050119, 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347420+0000", 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347421+0000", 2024-09-17T17:31:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.03e-07 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347462+0000", 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0862999999999998e-05 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347534+0000", 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2643000000000002e-05 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347550+0000", 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5633000000000001e-05 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347706+0000", 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015574400000000001 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.347737+0000", 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1164000000000002e-05 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350445+0000", 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027080540000000001 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:31.350470+0000", 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5715e-05 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.9s3 92/76 ECSubWrite(tid=213, reqid=osd.12.0:213, at_version=92'67, trim_to=0'0, roll_forward_to=91'65))", 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.682422884000001, 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049639100000000002, 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356159+0000", 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356183+0000", 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4491999999999999e-05 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356228+0000", 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5090999999999999e-05 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356237+0000", 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8950000000000004e-06 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356968+0000", 2024-09-17T17:31:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073084099999999998 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356978+0000", 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4069999999999999e-06 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.361091+0000", 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0041133680000000001 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.361123+0000", 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1605e-05 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 92/74 ECSubWrite(tid=128, reqid=osd.10.0:128, at_version=92'78, trim_to=0'0, roll_forward_to=85'76))", 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.682372792999999, 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031162680000000002, 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356209+0000", 2024-09-17T17:31:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0900000000000001e-07 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356229+0000", 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0058999999999999e-05 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356293+0000", 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4162e-05 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356301+0000", 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.401e-06 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356479+0000", 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000178297 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356488+0000", 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2259999999999994e-06 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.359287+0000", 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027985950000000001 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.359325+0000", 2024-09-17T17:31:04.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8318999999999997e-05 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.12s3 92/87 ECSubWrite(tid=86, reqid=osd.14.0:86, at_version=92'78, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.681864922999999, 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039887689999999996, 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356717+0000", 2024-09-17T17:31:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6300000000000005e-07 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356738+0000", 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1166e-05 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356812+0000", 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3659999999999996e-05 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.356820+0000", 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9319999999999996e-06 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.357010+0000", 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019038799999999999 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.357020+0000", 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7929999999999992e-06 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.360667+0000", 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003646935 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.360705+0000", 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8331999999999998e-05 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.3s4 96/74 ECSubWrite(tid=130, reqid=client.4566.0:925, at_version=96'80, trim_to=0'0, roll_forward_to=92'78))", 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.657985945, 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037922670000000002, 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.380596+0000", 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7100000000000001e-07 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.382960+0000", 2024-09-17T17:31:04.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002364395 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.382989+0000", 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8736000000000001e-05 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.382998+0000", 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7600000000000008e-06 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.383057+0000", 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9126000000000003e-05 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.383065+0000", 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0479999999999999e-06 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.384360+0000", 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012951690000000001 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:47.384388+0000", 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7861999999999999e-05 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs4 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:31:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.660245979000001, 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029972480000000001, 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378336+0000", 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378747+0000", 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041115500000000002 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378799+0000", 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1851000000000002e-05 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378806+0000", 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.435e-06 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378889+0000", 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2730000000000002e-05 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.378904+0000", 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4552000000000001e-05 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.381296+0000", 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023924250000000001 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.381333+0000", 2024-09-17T17:31:04.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6878000000000002e-05 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.18s3 100/75 ECSubWrite(tid=176, reqid=client.4566.0:964, at_version=100'31, trim_to=0'0, roll_forward_to=76'29))", 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:51.787785+0000", 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.250796258999999, 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057844150000000002, 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.132 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787785+0000", 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787785+0000", 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.787786+0000", 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.16e-07 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791036+0000", 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003250368 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791078+0000", 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1631000000000002e-05 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791087+0000", 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1970000000000002e-06 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791154+0000", 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6576999999999996e-05 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.791164+0000", 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0282000000000001e-05 2024-09-17T17:31:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.793526+0000", 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023622949999999999 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:51.793570+0000", 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3248999999999999e-05 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.137 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.137 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:04.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-17T17:31:04.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-8 --journal-path /var/lib/ceph/osd/ceph-8/journal --force --op remove --pgid 3.bs4 2024-09-17T17:31:04.247 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:31:04.247 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:04.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-17T17:31:04.363 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T17:31:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T17:31:04.364 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:04.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(osd.13.22:1 3.0s0 3:07ceeeba:::smithi10238019-36:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.795337587000006, 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017516434000000001, 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683025+0000", 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683026+0000", 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.195e-06 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683032+0000", 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4210000000000003e-06 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683042+0000", 2024-09-17T17:31:04.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683048+0000", 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4079999999999997e-06 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683111+0000", 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3219999999999994e-05 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.683117+0000", 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685006+0000", 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018886650000000001 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.685109+0000", 2024-09-17T17:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010286200000000001 2024-09-17T17:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:50.700541+0000", 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432603 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=44, last_complete=43'17, committed=1, applied=1))", 2024-09-17T17:31:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.764384363999994, 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034471605000000002, 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713978+0000", 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713979+0000", 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.713985+0000", 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9540000000000001e-06 2024-09-17T17:31:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748225+0000", 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034240591000000001 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748230+0000", 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4870000000000004e-06 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748294+0000", 2024-09-17T17:31:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4641000000000007e-05 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748450+0000", 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155377 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:302 3.9s0 3:93d1baed:::smithi10238019-5:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.732082720999998, 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.215598594, 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 302 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746280+0000", 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746282+0000", 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.75e-06 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746288+0000", 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746301+0000", 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2678e-05 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746311+0000", 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397000000000001e-05 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746376+0000", 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4552000000000001e-05 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746450+0000", 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4673999999999997e-05 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.746981+0000", 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053065700000000003 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.747716+0000", 2024-09-17T17:31:04.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073483100000000003 2024-09-17T17:31:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:31:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961789+0000", 2024-09-17T17:31:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21407319899999999 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961878+0000", 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9492000000000003e-05 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 43/35 ECSubWriteReply(tid=45, last_complete=43'19, committed=1, applied=1))", 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.730158219000003, 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.214078249, 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748204+0000", 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748205+0000", 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.748209+0000", 2024-09-17T17:31:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961602+0000", 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21339360600000001 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961614+0000", 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2157000000000001e-05 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.961695+0000", 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0520999999999997e-05 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:29:54.962283+0000", 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058771099999999998 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:410 3.9s0 3:93d1baed:::smithi10238019-5:head [copy-from ver 8] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.140262024999998, 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017989010999999999, 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 410 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338101+0000", 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338102+0000", 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5349999999999999e-06 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338107+0000", 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000003e-06 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338121+0000", 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3743e-05 2024-09-17T17:31:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338129+0000", 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4209999999999995e-06 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8885000000000003e-05 2024-09-17T17:31:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.338278+0000", 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0192999999999995e-05 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.349495+0000", 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011217173 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.350828+0000", 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013328700000000001 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356035+0000", 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052068000000000001 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:02.356090+0000", 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4731000000000002e-05 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 52/35 ECSubWriteReply(tid=76, last_complete=52'24, committed=1, applied=1))", 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.068810401, 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.18030790999999999, 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409552+0000", 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409553+0000", 2024-09-17T17:31:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4499999999999996e-07 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.409560+0000", 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589756+0000", 2024-09-17T17:31:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.180196197 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589765+0000", 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589834+0000", 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9301000000000006e-05 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:03.589860+0000", 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6058999999999999e-05 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:479 3.0s0 3:00c2d3b1:::smithi10238019-29:head [append 0~131072] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.130284877000001, 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017167436000000001, 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 479 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348078+0000", 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348079+0000", 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.719e-06 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348181+0000", 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101258 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348196+0000", 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5423000000000001e-05 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348204+0000", 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9510000000000006e-06 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348898+0000", 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069432499999999998 2024-09-17T17:31:04.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349042+0000", 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143848 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.349752+0000", 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071013599999999999 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351054+0000", 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001301618 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365180+0000", 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014125511 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365245+0000", 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5647000000000004e-05 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:480 3.0s0 3:00c2d3b1:::smithi10238019-29:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.130142509000002, 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017175950999999998, 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 480 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348220+0000", 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348221+0000", 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348225+0000", 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7969999999999998e-06 2024-09-17T17:31:04.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348235+0000", 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312000000000001e-05 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348241+0000", 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0340000000000002e-06 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350222+0000", 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019816209999999998 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350336+0000", 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113779 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.350754+0000", 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041740400000000002 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351686+0000", 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093241600000000004 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365336+0000", 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013650254000000001 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365396+0000", 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9704000000000003e-05 2024-09-17T17:31:04.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:481 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.130111427000003, 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018899091, 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 481 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348251+0000", 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348252+0000", 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348255+0000", 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6720000000000002e-06 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348265+0000", 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.132e-06 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.348271+0000", 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351236+0000", 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029650309999999999 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.351333+0000", 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7449999999999997e-05 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365440+0000", 2024-09-17T17:31:04.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106966 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.365519+0000", 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8703999999999997e-05 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:07.367150+0000", 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001631354 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.cs3 68/32 ECSubWrite(tid=191, reqid=client.4566.0:626, at_version=68'97, trim_to=0'0, roll_forward_to=68'94))", 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.978116446000001, 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018870181999999999, 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500246+0000", 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500247+0000", 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8299999999999996e-07 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500279+0000", 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2601000000000002e-05 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518513+0000", 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018233269 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518520+0000", 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2590000000000003e-06 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518590+0000", 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.518600+0000", 2024-09-17T17:31:04.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7329999999999998e-06 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519071+0000", 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047090800000000003 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.519116+0000", 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5521e-05 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWrite(3.12s1 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.977675411, 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84719745400000002, 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500687+0000", 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500688+0000", 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3e-07 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:19.500724+0000", 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-05 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346201+0000", 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84547703600000002 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346210+0000", 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7369999999999995e-06 2024-09-17T17:31:04.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346289+0000", 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8825999999999994e-05 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.346300+0000", 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1602e-05 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347848+0000", 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00154776 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347885+0000", 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6723000000000001e-05 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:651 3.0s0 3:07ceeeba:::smithi10238019-36:head [setxattr _header (57) in=7b] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:31:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.13105453, 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.090354326999999998, 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 651 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347308+0000", 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347309+0000", 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:31:04.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347313+0000", 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7270000000000001e-06 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347320+0000", 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1019999999999999e-06 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347330+0000", 2024-09-17T17:31:04.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0552e-05 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353490+0000", 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061600309999999998 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.353716+0000", 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000225987 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354187+0000", 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047076600000000001 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.355021+0000", 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083427900000000005 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437590+0000", 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.082568902999999999 2024-09-17T17:31:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437662+0000", 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2272000000000002e-05 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:652 3.0s0 3:07ceeeba:::smithi10238019-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.131022479999999, 2024-09-17T17:31:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.091356190000000004, 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 652 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347340+0000", 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347341+0000", 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347344+0000", 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0000000000000001e-06 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347352+0000", 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8250000000000005e-06 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.347357+0000", 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6489999999999996e-06 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354533+0000", 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007175781 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.354625+0000", 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2064999999999994e-05 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438014+0000", 2024-09-17T17:31:04.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.083388730999999994 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438102+0000", 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7861000000000005e-05 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.438696+0000", 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059467799999999996 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 69/67 ECSubWriteReply(tid=135, last_complete=69'58, committed=1, applied=1))", 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.122233573000003, 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081843374999999996, 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356129+0000", 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356130+0000", 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3600000000000003e-07 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.356133+0000", 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.794e-06 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437473+0000", 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.081339488000000001 2024-09-17T17:31:04.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437480+0000", 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6810000000000001e-06 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437533+0000", 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2958999999999999e-05 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:20.437972+0000", 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043981700000000001 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 73/72 ECSubReadReply(tid=170, attrs_read=0))", 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.445429877999999, 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.308285174, 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.032933+0000", 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.37e-07 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.033151+0000", 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000217491 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337311+0000", 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.304160612 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.337319+0000", 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4030000000000001e-06 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.340881+0000", 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035619649999999998 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.341218+0000", 2024-09-17T17:31:04.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033716600000000002 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpReadReply(3.9s0 74/72 ECSubReadReply(tid=172, attrs_read=0))", 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.123286444999998, 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64532973699999996, 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355076+0000", 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355077+0000", 2024-09-17T17:31:04.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:25.355304+0000", 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000086+0000", 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.64478191900000004 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000092+0000", 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1800000000000001e-06 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000163+0000", 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0883999999999994e-05 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.000406+0000", 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024273599999999999 2024-09-17T17:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPushReply(3.9s0 74/72 [PushReplyOp(3:96cbaefd:::smithi10238019-46:head)])", 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.465953804000002, 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32756439999999998, 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012409+0000", 2024-09-17T17:31:04.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.012412+0000", 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7250000000000002e-06 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337269+0000", 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.32485740099999999 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.337283+0000", 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3655999999999999e-05 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339351+0000", 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020684850000000001 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:26.339973+0000", 2024-09-17T17:31:04.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062179200000000003 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 89/75 ECSubWriteReply(tid=200, last_complete=89'71, committed=1, applied=1))", 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.759616837999999, 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91509105899999998, 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718746+0000", 2024-09-17T17:31:04.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:40.718747+0000", 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.243e-06 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633701+0000", 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91495358800000004 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633709+0000", 2024-09-17T17:31:04.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633787+0000", 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7503999999999995e-05 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:41.633837+0000", 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0012000000000002e-05 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4566.0:911 3.0s0 3:00c2d3b1:::smithi10238019-29:head [read 822626~1028766,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.105619642000001, 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.94643117499999996, 2024-09-17T17:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4566", 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 911 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372743+0000", 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372744+0000", 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6000000000000003e-07 2024-09-17T17:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372746+0000", 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372758+0000", 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999e-05 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.372764+0000", 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5289999999999999e-06 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.373482+0000", 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071808600000000005 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:42.373547+0000", 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5382000000000005e-05 2024-09-17T17:31:04.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:43.319174+0000", 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.94562681400000004 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 97/75 ECSubWriteReply(tid=225, last_complete=97'88, committed=1, applied=1))", 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.103606629000002, 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21796726999999999, 2024-09-17T17:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T17:31:04.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374756+0000", 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6800000000000001e-07 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.374758+0000", 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7379999999999999e-06 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T17:31:04.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592512+0000", 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.21775348 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592527+0000", 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5659999999999999e-05 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592649+0000", 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121579 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T17:31:04.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T17:30:48.592723+0000", 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4444999999999996e-05 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T17:31:04.586 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:04.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T17:31:04.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T17:31:04.709 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:31:04.709 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:04.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T17:31:04.827 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:31:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:31:04.827 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:04.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T17:31:04.944 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:31:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:31:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:31:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s5 36/35 ECSubWrite(tid=12, reqid=client.4566.0:148, at_version=36'9, trim_to=0'0, roll_forward_to=36'6))", 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 76.464734617000005, 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.51067298000000005, 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476784+0000", 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.476798+0000", 2024-09-17T17:31:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.367e-05 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986749+0000", 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.50995120999999999 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986759+0000", 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5689999999999994e-06 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986853+0000", 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.3851000000000007e-05 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.986867+0000", 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4664999999999999e-05 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987427+0000", 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00056014300000000003 2024-09-17T17:31:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.987457+0000", 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9142999999999999e-05 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 37/35 ECSubWriteReply(tid=9, last_complete=37'6, committed=1, applied=1))", 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 75.946468378999995, 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054635318000000002, 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995050+0000", 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995051+0000", 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2799999999999995e-07 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995055+0000", 2024-09-17T17:31:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1019999999999999e-06 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049558+0000", 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.054503348 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049567+0000", 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2310000000000002e-06 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049637+0000", 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9647999999999995e-05 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:49.049685+0000", 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8260999999999998e-05 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=11, last_complete=39'8, committed=1, applied=1))", 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 74.254750856000001, 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021877058000000001, 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686767+0000", 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686768+0000", 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.06e-07 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.686775+0000", 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3390000000000004e-06 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708283+0000", 2024-09-17T17:31:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.021507419 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708291+0000", 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8150000000000006e-06 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708610+0000", 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000319248 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708644+0000", 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4631000000000001e-05 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:204 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~376832] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 74.235307574000004, 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0099436439999999997, 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 204 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:31:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706211+0000", 2024-09-17T17:31:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706212+0000", 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.294e-06 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706551+0000", 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000339327 2024-09-17T17:31:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706567+0000", 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5648e-05 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706576+0000", 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8119999999999997e-06 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706639+0000", 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3362999999999996e-05 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706704+0000", 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5197999999999995e-05 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707194+0000", 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00048943900000000004 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708376+0000", 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011827789999999999 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716077+0000", 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0077002609999999999 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.716154+0000", 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7522999999999996e-05 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:205 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:31:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 74.234920670999998, 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014541866000000001, 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 205 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706598+0000", 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0100000000000003e-07 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706603+0000", 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1280000000000001e-06 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706612+0000", 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.285e-06 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706618+0000", 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0870000000000003e-06 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707569+0000", 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095131700000000003 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707666+0000", 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6947000000000005e-05 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708113+0000", 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044667600000000002 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.709038+0000", 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00092496799999999997 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721070+0000", 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012032568 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721139+0000", 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8989e-05 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:206 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 74.234887303999997, 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015826837, 2024-09-17T17:31:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 206 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706631+0000", 2024-09-17T17:31:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5199999999999997e-07 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706634+0000", 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7480000000000002e-06 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706642+0000", 2024-09-17T17:31:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.011e-06 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.706648+0000", 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7880000000000003e-06 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708828+0000", 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002179589 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.708914+0000", 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6273999999999994e-05 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721437+0000", 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012522688000000001 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721573+0000", 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013612100000000001 2024-09-17T17:31:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.722458+0000", 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000885066 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 39/35 ECSubWriteReply(tid=14, last_complete=39'11, committed=1, applied=1))", 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:31:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 74.230437832999996, 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010321001, 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:31:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711080+0000", 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711081+0000", 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3200000000000005e-07 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.711084+0000", 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4759999999999998e-06 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720961+0000", 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0098766070000000008 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.720966+0000", 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8640000000000004e-06 2024-09-17T17:31:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721013+0000", 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7460999999999998e-05 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.721401+0000", 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00038806099999999998 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:367 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~589824] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 64.602827017999999, 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.012183095999999999, 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 367 2024-09-17T17:31:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338691+0000", 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.338693+0000", 2024-09-17T17:31:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.795e-06 2024-09-17T17:31:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341609+0000", 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029156210000000002 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341623+0000", 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4583000000000001e-05 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341630+0000", 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2230000000000001e-06 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341691+0000", 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0878000000000002e-05 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.341771+0000", 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9308999999999996e-05 2024-09-17T17:31:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343069+0000", 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012979420000000001 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348577+0000", 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0055080229999999999 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350781+0000", 2024-09-17T17:31:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002204742 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.350874+0000", 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2979999999999994e-05 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:368 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~573440] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:31:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 64.599187001000004, 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0088382370000000005, 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 368 2024-09-17T17:31:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342331+0000", 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342332+0000", 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.08e-06 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.342959+0000", 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062686600000000001 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343062+0000", 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010252 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343083+0000", 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1135e-05 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344260+0000", 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011769969999999999 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.344389+0000", 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012922399999999999 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.345712+0000", 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001323182 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.348773+0000", 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030611649999999998 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351091+0000", 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023171590000000001 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351169+0000", 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8909e-05 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:370 3.4s0 3:26d49efd:::smithi10238019-44:head [setxattr _header (56) in=7b] snapc a=[a,9,8,7,6] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 64.597817548999998, 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0082819799999999996, 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 370 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343701+0000", 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3799999999999997e-07 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343710+0000", 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2570000000000007e-06 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343715+0000", 2024-09-17T17:31:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6829999999999996e-06 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343721+0000", 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8810000000000001e-06 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349122+0000", 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.005400841 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349229+0000", 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107076 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.349735+0000", 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00050621599999999998 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351781+0000", 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020453279999999999 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351908+0000", 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012790499999999999 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.351983+0000", 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.4154999999999997e-05 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:371 3.4s0 3:26d49efd:::smithi10238019-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 64.597790505999996, 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010270708, 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 371 2024-09-17T17:31:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343728+0000", 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.4799999999999999e-07 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343730+0000", 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4389999999999999e-06 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343735+0000", 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1919999999999998e-06 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.343738+0000", 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1360000000000001e-06 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352473+0000", 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0087356429999999995 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.352570+0000", 2024-09-17T17:31:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6261999999999994e-05 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:00.353998+0000", 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001428688 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.fs6 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:31:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.561140262999999, 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75327243099999996, 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380378+0000", 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380379+0000", 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8100000000000002e-07 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.380396+0000", 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7717e-05 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132975+0000", 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.75257808000000004 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.132981+0000", 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2249999999999997e-06 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133046+0000", 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4944999999999996e-05 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133053+0000", 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133623+0000", 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057078100000000002 2024-09-17T17:31:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.133650+0000", 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6996e-05 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:495 3.4s0 3:26d49efd:::smithi10238019-44:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 56.283454134000003, 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.022462405000000001, 2024-09-17T17:31:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 495 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658064+0000", 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658065+0000", 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2600000000000001e-07 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658072+0000", 2024-09-17T17:31:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8430000000000004e-06 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658082+0000", 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.927e-06 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.658089+0000", 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8909999999999998e-06 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660406+0000", 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0023178890000000001 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.660504+0000", 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7341000000000002e-05 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661132+0000", 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00062810899999999996 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661861+0000", 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072939299999999995 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680403+0000", 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018541588000000001 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680526+0000", 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012359799999999999 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/35 ECSubWriteReply(tid=37, last_complete=57'27, committed=1, applied=1))", 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 56.279687308, 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019253007999999999, 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661831+0000", 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661832+0000", 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.661839+0000", 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7999999999999999e-06 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680112+0000", 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018272335000000001 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680126+0000", 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3867e-05 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.680275+0000", 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000149741 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.681084+0000", 2024-09-17T17:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00080847500000000001 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=196, reqid=client.4566.0:639, at_version=68'102, trim_to=0'0, roll_forward_to=68'97))", 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 45.415781871, 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.667795892, 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525736+0000", 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525737+0000", 2024-09-17T17:31:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9299999999999996e-07 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.525756+0000", 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9026999999999999e-05 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192770+0000", 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.66701338300000002 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192793+0000", 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3203000000000001e-05 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192903+0000", 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109973 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.192912+0000", 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9139999999999997e-06 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193502+0000", 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00058984700000000001 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193532+0000", 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0651999999999997e-05 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 68/32 ECSubWrite(tid=197, reqid=client.4566.0:639, at_version=0'0, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.747609373000003, 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15173107799999999, 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193909+0000", 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5799999999999999e-07 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.193938+0000", 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8988000000000001e-05 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344048+0000", 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15010903 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344059+0000", 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1148000000000001e-05 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344595+0000", 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00053675200000000004 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.344608+0000", 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2211e-05 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345601+0000", 2024-09-17T17:31:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00099356800000000001 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:20.345640+0000", 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8723000000000002e-05 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "replica_scrub(pg: 3.16s4,from:0'0,to:83'146,epoch:84/74,start:3:68000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.019858236000001, 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.13087489099999999, 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:31:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921660+0000", 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921661+0000", 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1230000000000001e-06 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:35.921668+0000", 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5470000000000002e-06 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052265+0000", 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.130597518 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052281+0000", 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5537e-05 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052444+0000", 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000163454 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052471+0000", 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6979999999999999e-05 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.052535+0000", 2024-09-17T17:31:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3732000000000006e-05 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 91/32 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=91'144, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345231+0000", 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.596286798000001, 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0094121650000000001, 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345231+0000", 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345231+0000", 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345232+0000", 2024-09-17T17:31:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2100000000000001e-07 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345260+0000", 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8119e-05 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345309+0000", 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9007e-05 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345319+0000", 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0522e-05 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.347887+0000", 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002567897 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.347905+0000", 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7796e-05 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354579+0000", 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0066740000000000002 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354644+0000", 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4603000000000005e-05 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs5 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.346888+0000", 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.594629732000001, 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.010283685000000001, 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346888+0000", 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346888+0000", 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346889+0000", 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6700000000000002e-07 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346916+0000", 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.6548999999999999e-05 2024-09-17T17:31:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346966+0000", 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0084999999999999e-05 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346976+0000", 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.859e-06 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348683+0000", 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001707124 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348695+0000", 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2055000000000001e-05 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.357139+0000", 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0084439010000000002 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.357172+0000", 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3145000000000002e-05 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/94 ECSubWriteReply(tid=94, last_complete=95'60, committed=1, applied=1))", 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:46.358704+0000", 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 18.582813768000001, 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014087463, 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358704+0000", 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358704+0000", 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358705+0000", 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8899999999999998e-07 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.358711+0000", 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1210000000000003e-06 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.371995+0000", 2024-09-17T17:31:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.013283483 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.372008+0000", 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.344e-05 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.372159+0000", 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00015027399999999999 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:46.372792+0000", 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00063325600000000001 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.050 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T17:31:05.160 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:31:05.160 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T17:31:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:31:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:31:05.275 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T17:31:05.392 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:31:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:31:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/35 ECSubWrite(tid=15, reqid=client.4566.0:164, at_version=37'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 76.397917145999998, 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033373299999999999, 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991766+0000", 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6199999999999997e-07 2024-09-17T17:31:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.991794+0000", 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7951999999999999e-05 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993932+0000", 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0021380000000000001 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.993941+0000", 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8489999999999995e-06 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994049+0000", 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000107896 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.994058+0000", 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:31:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995071+0000", 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010122849999999999 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:48.995103+0000", 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2434999999999997e-05 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 42/32 ECSubWrite(tid=34, reqid=client.4566.0:280, at_version=42'25, trim_to=0'0, roll_forward_to=41'21))", 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 71.663844105999999, 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85910165100000002, 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:31:05.401 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725839+0000", 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725840+0000", 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1800000000000004e-07 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.402 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:53.725864+0000", 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4119000000000001e-05 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583977+0000", 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.85811290799999995 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.583995+0000", 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8756000000000001e-05 2024-09-17T17:31:05.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584143+0000", 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014752600000000001 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584154+0000", 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1319e-05 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.404 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584893+0000", 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000738851 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.584940+0000", 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7253999999999999e-05 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 43/32 ECSubWrite(tid=19, reqid=osd.7.0:19, at_version=43'16, trim_to=0'0, roll_forward_to=42'12))", 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 70.695816234000006, 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002907125, 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693867+0000", 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693868+0000", 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.7199999999999997e-07 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.407 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693893+0000", 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5100999999999999e-05 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693943+0000", 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0862999999999997e-05 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.693953+0000", 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8819999999999996e-06 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694024+0000", 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0504000000000001e-05 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694035+0000", 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696736+0000", 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027008470000000001 2024-09-17T17:31:05.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696774+0000", 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7687e-05 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=38, reqid=osd.8.0:38, at_version=43'29, trim_to=0'0, roll_forward_to=42'25))", 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 70.695574124000004, 2024-09-17T17:31:05.410 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028702689999999999, 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694109+0000", 2024-09-17T17:31:05.411 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694110+0000", 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6899999999999996e-07 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694139+0000", 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9669999999999999e-05 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694185+0000", 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5447e-05 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694194+0000", 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6420000000000002e-06 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.413 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694259+0000", 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4159999999999996e-05 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.694269+0000", 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0324e-05 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696936+0000", 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026667359999999998 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.414 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.696979+0000", 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3321000000000001e-05 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.005082307000002, 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029370899999999998, 2024-09-17T17:31:05.415 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384600+0000", 2024-09-17T17:31:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.384601+0000", 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8e-07 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385325+0000", 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00072435200000000005 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.417 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385406+0000", 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.0667999999999994e-05 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385415+0000", 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385509+0000", 2024-09-17T17:31:05.418 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4137999999999999e-05 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385519+0000", 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0208e-05 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387496+0000", 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001976387 2024-09-17T17:31:05.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.387538+0000", 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1996999999999998e-05 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.004354145000001, 2024-09-17T17:31:05.420 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029662, 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:31:05.421 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385329+0000", 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385362+0000", 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2403999999999997e-05 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.422 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385471+0000", 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000109485 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.385479+0000", 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6750000000000002e-06 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.423 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386169+0000", 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068989099999999999 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386179+0000", 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0278000000000001e-05 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388245+0000", 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020665150000000001 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.388295+0000", 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9369000000000001e-05 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 45.950607044999998, 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029678690000000001, 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439076+0000", 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1899999999999998e-07 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439101+0000", 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4655e-05 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439183+0000", 2024-09-17T17:31:05.427 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2472000000000006e-05 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439193+0000", 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0063e-05 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439257+0000", 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3775000000000005e-05 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.428 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.439270+0000", 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2588e-05 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.441995+0000", 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002725505 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442044+0000", 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8392e-05 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.8s4 70/67 ECSubWrite(tid=155, reqid=osd.6.0:155, at_version=70'27, trim_to=0'0, roll_forward_to=68'25))", 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.044514374000002, 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00308939, 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345168+0000", 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.431 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345169+0000", 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345192+0000", 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3241e-05 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345239+0000", 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6607999999999998e-05 2024-09-17T17:31:05.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345247+0000", 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5339999999999999e-06 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345711+0000", 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00046389400000000001 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.433 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345720+0000", 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0710000000000001e-06 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348210+0000", 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024892099999999999 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348258+0000", 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8208000000000003e-05 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.434 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=199, reqid=osd.11.0:199, at_version=70'104, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.044220367000001, 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002830782, 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.435 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345462+0000", 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345463+0000", 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8300000000000001e-07 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345486+0000", 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3453999999999999e-05 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345559+0000", 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2760000000000001e-05 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345586+0000", 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.7670000000000001e-05 2024-09-17T17:31:05.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345650+0000", 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3139999999999995e-05 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.345661+0000", 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1342000000000001e-05 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348252+0000", 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0025906829999999999 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348293+0000", 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.155e-05 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/32 ECSubWrite(tid=200, reqid=osd.11.0:200, at_version=70'106, trim_to=0'0, roll_forward_to=68'102))", 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.043638932999997, 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027884030000000001, 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:31:05.440 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346044+0000", 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346045+0000", 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0419999999999999e-06 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.441 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346081+0000", 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5942999999999998e-05 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346125+0000", 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3949999999999998e-05 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346134+0000", 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5019999999999998e-06 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346909+0000", 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00077504099999999997 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.346921+0000", 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1598e-05 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348789+0000", 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001868268 2024-09-17T17:31:05.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.348832+0000", 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3059000000000002e-05 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 24.046281885999999, 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003502827, 2024-09-17T17:31:05.444 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343401+0000", 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9999999999999999e-07 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343433+0000", 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2026999999999997e-05 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343478+0000", 2024-09-17T17:31:05.446 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4561999999999998e-05 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343489+0000", 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1269e-05 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343600+0000", 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000110818 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.447 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343617+0000", 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.6824999999999999e-05 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346851+0000", 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032341750000000002 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346904+0000", 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2850999999999999e-05 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.448 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 91/32 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=91'144, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.044841963, 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004955597, 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.449 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344841+0000", 2024-09-17T17:31:05.450 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5300000000000004e-07 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344871+0000", 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0083e-05 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.344995+0000", 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012343400000000001 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.451 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345025+0000", 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0437999999999999e-05 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345098+0000", 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3114999999999994e-05 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345110+0000", 2024-09-17T17:31:05.452 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2004999999999999e-05 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349747+0000", 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046367179999999997 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349796+0000", 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9251e-05 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 91/32 ECSubWrite(tid=101, reqid=osd.7.0:101, at_version=91'83, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.044165463999999, 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0081096810000000005, 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.454 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345517+0000", 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7499999999999999e-07 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.455 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345541+0000", 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3065999999999999e-05 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345608+0000", 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7149000000000004e-05 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.456 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345617+0000", 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2839999999999995e-06 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345692+0000", 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5053000000000001e-05 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.457 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345703+0000", 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1172000000000001e-05 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353581+0000", 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0078780229999999996 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353627+0000", 2024-09-17T17:31:05.458 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5759e-05 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds6 91/32 ECSubWrite(tid=102, reqid=osd.7.0:102, at_version=91'85, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.044139393999998, 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0089760780000000002, 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.459 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345543+0000", 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.460 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345544+0000", 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.75e-07 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345564+0000", 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0519000000000001e-05 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348824+0000", 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032595050000000002 2024-09-17T17:31:05.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348834+0000", 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0329000000000001e-05 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349653+0000", 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000818445 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.462 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349662+0000", 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5119999999999997e-06 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354468+0000", 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004805363 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354519+0000", 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1929999999999999e-05 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.463 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.040816695, 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003290444, 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.464 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348866+0000", 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348867+0000", 2024-09-17T17:31:05.465 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.9899999999999999e-07 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348892+0000", 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5429e-05 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348942+0000", 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9731999999999998e-05 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.466 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348950+0000", 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3350000000000007e-06 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349267+0000", 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00031690899999999998 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349280+0000", 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2880999999999999e-05 2024-09-17T17:31:05.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352124+0000", 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028441819999999998 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352157+0000", 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2177e-05 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.es6 91/32 ECSubWrite(tid=75, reqid=client.4566.0:895, at_version=91'58, trim_to=0'0, roll_forward_to=83'57))", 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.006279414000002, 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0040288700000000004, 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.469 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383403+0000", 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.383404+0000", 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6399999999999998e-07 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.470 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385486+0000", 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020826339999999999 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385523+0000", 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6739999999999997e-05 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385534+0000", 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0937e-05 2024-09-17T17:31:05.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385626+0000", 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.2428000000000003e-05 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.385655+0000", 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8343e-05 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.472 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.387395+0000", 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001740157 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.387432+0000", 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7267000000000001e-05 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 92/32 ECSubWrite(tid=282, reqid=osd.11.0:282, at_version=92'148, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:31:05.473 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.03970713, 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029263319999999998, 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.474 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.349976+0000", 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4999999999999999e-07 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350009+0000", 2024-09-17T17:31:05.475 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2889999999999999e-05 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350288+0000", 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00027886300000000002 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350299+0000", 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1931e-05 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350372+0000", 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2140999999999999e-05 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350386+0000", 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4194e-05 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.477 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352851+0000", 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0024656330000000001 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352902+0000", 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0529999999999999e-05 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 92/32 ECSubWrite(tid=283, reqid=osd.11.0:283, at_version=92'150, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:31:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.039670986000001, 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0029327540000000001, 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.479 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350012+0000", 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4999999999999998e-07 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350038+0000", 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5956e-05 2024-09-17T17:31:05.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350867+0000", 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00082833600000000005 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350881+0000", 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4865000000000001e-05 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.481 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350949+0000", 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7794999999999997e-05 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.350960+0000", 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1083e-05 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352909+0000", 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019484750000000001 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352945+0000", 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5794000000000003e-05 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs2 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.011714226999999, 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0037244769999999999, 2024-09-17T17:31:05.483 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377969+0000", 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6499999999999999e-07 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378414+0000", 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044483400000000002 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.485 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378460+0000", 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5980999999999999e-05 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378468+0000", 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3049999999999993e-06 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378582+0000", 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000114036 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.486 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378597+0000", 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4647000000000001e-05 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.381637+0000", 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003039698 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.381693+0000", 2024-09-17T17:31:05.487 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6410999999999999e-05 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.18s4 100/75 ECSubWrite(tid=176, reqid=client.4566.0:964, at_version=100'31, trim_to=0'0, roll_forward_to=76'29))", 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:51.788648+0000", 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 13.601034507, 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0054892819999999998, 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.488 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.788648+0000", 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.788648+0000", 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.489 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.788649+0000", 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5499999999999997e-07 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792114+0000", 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0034652390000000002 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792139+0000", 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4669999999999999e-05 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792145+0000", 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8209999999999998e-06 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792202+0000", 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.7556999999999998e-05 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.792209+0000", 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9059999999999996e-06 2024-09-17T17:31:05.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.794100+0000", 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001891085 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.794138+0000", 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7149e-05 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.493 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.493 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-17T17:31:05.604 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:05.604 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:05.604 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:31:05.604 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-17T17:31:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T17:31:05.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T17:31:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: marking collection for removal 2024-09-17T17:31:05.663 INFO:teuthology.orchestra.run.smithi007.stdout:setting '_remove' omap key 2024-09-17T17:31:05.663 INFO:teuthology.orchestra.run.smithi007.stdout:finish_remove_pgs 3.bs4_head removing 3.bs4 2024-09-17T17:31:05.664 INFO:teuthology.orchestra.run.smithi007.stdout:Remove successful 2024-09-17T17:31:05.718 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:05.718 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:05.718 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:31:05.718 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:31:05.718 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.493210120000001, 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032487919999999999, 2024-09-17T17:31:05.835 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339224+0000", 2024-09-17T17:31:05.836 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339225+0000", 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6799999999999999e-07 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339248+0000", 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3654000000000001e-05 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.837 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339288+0000", 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9734999999999998e-05 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339297+0000", 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6649999999999992e-06 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339672+0000", 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037528099999999999 2024-09-17T17:31:05.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339682+0000", 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0168e-05 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342436+0000", 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027540889999999999 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.839 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342473+0000", 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6331999999999997e-05 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 56/32 ECSubWrite(tid=124, reqid=osd.11.0:124, at_version=56'50, trim_to=0'0, roll_forward_to=53'48))", 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.493032020999998, 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030245340000000002, 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339402+0000", 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0100000000000001e-07 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339459+0000", 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6342999999999999e-05 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339692+0000", 2024-09-17T17:31:05.842 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00023318199999999999 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339701+0000", 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7469999999999994e-06 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339766+0000", 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5277999999999994e-05 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.843 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.339776+0000", 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8439999999999993e-06 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342401+0000", 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026253349999999999 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.342426+0000", 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5503999999999998e-05 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 56/35 ECSubWrite(tid=92, reqid=osd.12.0:92, at_version=56'40, trim_to=0'0, roll_forward_to=56'34))", 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 58.478522517999998, 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003074832, 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.845 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353911+0000", 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353912+0000", 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:31:05.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.353937+0000", 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5015e-05 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354085+0000", 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014835900000000001 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.847 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.354091+0000", 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8220000000000003e-06 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356082+0000", 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0019913650000000001 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356088+0000", 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1190000000000002e-06 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.848 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356953+0000", 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00086492399999999999 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:07.356986+0000", 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2994999999999998e-05 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.849 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=136, reqid=osd.11.0:136, at_version=57'54, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.459981020000001, 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030867720000000002, 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372453+0000", 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2800000000000001e-07 2024-09-17T17:31:05.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372483+0000", 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9967999999999998e-05 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372565+0000", 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2091999999999999e-05 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.852 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372574+0000", 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8869999999999998e-06 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372659+0000", 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5358999999999994e-05 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.372674+0000", 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4827000000000001e-05 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375499+0000", 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028244870000000001 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375540+0000", 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1023999999999999e-05 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.854 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 57/32 ECSubWrite(tid=139, reqid=osd.11.0:139, at_version=57'56, trim_to=0'0, roll_forward_to=56'52))", 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.458730346999999, 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0033844539999999998, 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373704+0000", 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8999999999999995e-07 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373730+0000", 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5689000000000001e-05 2024-09-17T17:31:05.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373771+0000", 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1208000000000003e-05 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374141+0000", 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00037011200000000002 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374273+0000", 2024-09-17T17:31:05.857 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013144900000000001 2024-09-17T17:31:05.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.858 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.858 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374283+0000", 2024-09-17T17:31:05.858 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0147e-05 2024-09-17T17:31:05.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377050+0000", 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002767346 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377088+0000", 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7713000000000003e-05 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/32 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=68'92, trim_to=0'0, roll_forward_to=66'90))", 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.391834101000001, 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003708654, 2024-09-17T17:31:05.860 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440600+0000", 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5499999999999999e-07 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440633+0000", 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2728999999999998e-05 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440769+0000", 2024-09-17T17:31:05.862 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013610200000000001 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.440779+0000", 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9410000000000002e-06 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442220+0000", 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0014405760000000001 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.863 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.442231+0000", 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1847e-05 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444268+0000", 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020363299999999998 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444309+0000", 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0774e-05 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.864 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 68/67 ECSubWrite(tid=122, reqid=client.4566.0:616, at_version=68'46, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.388361037999999, 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030735390000000001, 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.865 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444073+0000", 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444074+0000", 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1200000000000002e-07 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444177+0000", 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010335700000000001 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444221+0000", 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4370999999999999e-05 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444232+0000", 2024-09-17T17:31:05.867 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0178e-05 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444294+0000", 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2204000000000003e-05 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.444303+0000", 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9630000000000004e-06 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.868 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447110+0000", 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002806809 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.447146+0000", 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6944999999999999e-05 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.869 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=157, to_read={3:00c2d3b1:::smithi10238019-29:head=0,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 41.017151669, 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30626450500000002, 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.870 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815282+0000", 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815283+0000", 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.9299999999999998e-07 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:24.815290+0000", 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.8709999999999999e-06 2024-09-17T17:31:05.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121092+0000", 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.30580278100000002 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121102+0000", 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5170000000000005e-06 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.872 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121186+0000", 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4229999999999998e-05 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.121547+0000", 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00036061300000000003 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpRead(3.0s6 73/72 ECSubRead(tid=171, to_read={3:00c2d3b1:::smithi10238019-29:head=262144,262144,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:31:05.873 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 40.708314176000002, 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.21319038800000001, 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:31:05.874 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124120+0000", 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.61e-07 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.124129+0000", 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5269999999999998e-06 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.875 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336658+0000", 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.212529417 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336671+0000", 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2658e-05 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.336735+0000", 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4203999999999997e-05 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:25.337310+0000", 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00057492100000000005 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 85/32 ECSubWrite(tid=83, reqid=osd.7.0:83, at_version=85'73, trim_to=0'0, roll_forward_to=83'71))", 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.485890676, 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0028233020000000002, 2024-09-17T17:31:05.877 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346543+0000", 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346544+0000", 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3099999999999999e-07 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346574+0000", 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0852999999999998e-05 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346642+0000", 2024-09-17T17:31:05.879 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.7183999999999999e-05 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346653+0000", 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1076e-05 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.880 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346728+0000", 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5459999999999999e-05 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.346741+0000", 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2894999999999999e-05 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.349319+0000", 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00257771 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.349367+0000", 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7793000000000001e-05 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 90/32 ECSubWrite(tid=97, reqid=client.4566.0:884, at_version=90'79, trim_to=0'0, roll_forward_to=89'77))", 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 24.488991480999999, 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0030163410000000001, 2024-09-17T17:31:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343443+0000", 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2100000000000001e-07 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343467+0000", 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4273e-05 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343522+0000", 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5386999999999997e-05 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343534+0000", 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.198e-05 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343611+0000", 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.6996999999999995e-05 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.343623+0000", 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2194999999999999e-05 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346410+0000", 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002786602 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.346459+0000", 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.8785999999999997e-05 2024-09-17T17:31:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.4s2 90/74 ECSubWrite(tid=76, reqid=client.4566.0:887, at_version=90'58, trim_to=0'0, roll_forward_to=83'56))", 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:41.344904+0000", 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 24.487529736999999, 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0035865720000000001, 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.344904+0000", 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.344904+0000", 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.344905+0000", 2024-09-17T17:31:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.9200000000000002e-07 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345022+0000", 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00011770899999999999 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345095+0000", 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.2934000000000003e-05 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345106+0000", 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0342e-05 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345191+0000", 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5152000000000001e-05 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.345202+0000", 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1195999999999999e-05 2024-09-17T17:31:05.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.348449+0000", 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032475669999999998 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:41.348491+0000", 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.138e-05 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 91/32 ECSubWrite(tid=101, reqid=osd.7.0:101, at_version=91'83, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.486797025000001, 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004051476, 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:31:05.892 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345637+0000", 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3000000000000002e-07 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.893 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345666+0000", 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8405999999999999e-05 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345755+0000", 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8998000000000004e-05 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345764+0000", 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.38e-06 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345859+0000", 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.4679000000000006e-05 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345871+0000", 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2084e-05 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349647+0000", 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003776106 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.349688+0000", 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1492999999999997e-05 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.ds4 91/32 ECSubWrite(tid=102, reqid=osd.7.0:102, at_version=91'85, trim_to=0'0, roll_forward_to=90'81))", 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.486765172999998, 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0061800359999999999, 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.898 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345669+0000", 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6699999999999999e-07 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345715+0000", 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5666e-05 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.899 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345800+0000", 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4781e-05 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.345807+0000", 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9380000000000003e-06 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.900 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348723+0000", 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00291647 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.348733+0000", 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9459999999999993e-06 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351805+0000", 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003071823 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351849+0000", 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3945e-05 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 91/32 ECSubWrite(tid=280, reqid=osd.11.0:280, at_version=91'146, trim_to=0'0, roll_forward_to=89'142))", 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.346604+0000", 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.485830169, 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046338799999999999, 2024-09-17T17:31:05.902 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346604+0000", 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346604+0000", 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346605+0000", 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.879999999999999e-07 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346630+0000", 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4813000000000001e-05 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.904 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346676+0000", 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6278e-05 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346686+0000", 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0409e-05 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346782+0000", 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.5859999999999994e-05 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.905 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.346794+0000", 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1396999999999999e-05 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351196+0000", 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044024199999999998 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351238+0000", 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1715000000000003e-05 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.0s6 91/75 ECSubWrite(tid=202, reqid=osd.12.0:202, at_version=91'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 23.481299938999999, 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003643519, 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351134+0000", 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9099999999999999e-07 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.351171+0000", 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6375999999999999e-05 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352114+0000", 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00094281599999999997 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.352122+0000", 2024-09-17T17:31:05.909 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5609999999999999e-06 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353080+0000", 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00095752200000000004 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.353090+0000", 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.078e-05 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.910 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354718+0000", 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001627229 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:42.354778+0000", 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9843999999999998e-05 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.911 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 92/32 ECSubWrite(tid=282, reqid=osd.11.0:282, at_version=92'148, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.352273+0000", 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.480160874999999, 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0031782680000000002, 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352273+0000", 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.912 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352273+0000", 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352274+0000", 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.2600000000000002e-07 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352304+0000", 2024-09-17T17:31:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0154e-05 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352477+0000", 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017361300000000001 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352487+0000", 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6949999999999995e-06 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.914 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.353396+0000", 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00090919799999999995 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.353406+0000", 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0098999999999999e-05 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.355413+0000", 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0020068479999999999 2024-09-17T17:31:05.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.355451+0000", 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8034999999999998e-05 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 92/32 ECSubWrite(tid=283, reqid=osd.11.0:283, at_version=92'150, trim_to=0'0, roll_forward_to=91'146))", 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 22.480127089, 2024-09-17T17:31:05.916 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039958600000000004, 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:31:05.917 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352307+0000", 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.2599999999999998e-07 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352343+0000", 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5757999999999999e-05 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.918 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352507+0000", 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016381600000000001 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.352515+0000", 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.3350000000000007e-06 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.354318+0000", 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0018024930000000001 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.354329+0000", 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0991e-05 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.356257+0000", 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001928683 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:43.356303+0000", 2024-09-17T17:31:05.920 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5358000000000002e-05 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.cs6 97/32 ECSubWrite(tid=336, reqid=client.4566.0:941, at_version=97'156, trim_to=0'0, roll_forward_to=93'154))", 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 17.454453334, 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0042165780000000003, 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.921 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.377981+0000", 2024-09-17T17:31:05.922 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5200000000000001e-07 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378428+0000", 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00044723199999999999 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378473+0000", 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5019999999999999e-05 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.923 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378492+0000", 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.8980000000000001e-05 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378563+0000", 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1007999999999995e-05 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.378574+0000", 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0735e-05 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.382145+0000", 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003571482 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:48.382197+0000", 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.1969000000000003e-05 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWrite(3.9s5 100/76 ECSubWrite(tid=237, reqid=client.4566.0:969, at_version=100'75, trim_to=0'0, roll_forward_to=100'71))", 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:51.772592+0000", 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 14.059842263, 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0046045579999999999, 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.772592+0000", 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.772592+0000", 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.772606+0000", 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3933e-05 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:05.927 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774378+0000", 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001772266 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774410+0000", 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2135999999999999e-05 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774420+0000", 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.6039999999999999e-06 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774473+0000", 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.3627e-05 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.774483+0000", 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.011e-05 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.777165+0000", 2024-09-17T17:31:05.929 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002681606 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:51.777196+0000", 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1275999999999997e-05 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:05.930 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:05.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-17T17:31:06.039 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:06.039 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:06.039 INFO:teuthology.orchestra.run.smithi026.stdout: "num_ops": 0 2024-09-17T17:31:06.039 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:06.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-17T17:31:06.155 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:06.155 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [], 2024-09-17T17:31:06.155 INFO:teuthology.orchestra.run.smithi026.stdout: "complaint_time": 30, 2024-09-17T17:31:06.155 INFO:teuthology.orchestra.run.smithi026.stdout: "num_blocked_ops": 0 2024-09-17T17:31:06.155 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:06.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-17T17:31:06.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-8 --journal-path /var/lib/ceph/osd/ceph-8/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.bs4.8 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout: "size": 20, 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 600, 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout: "ops": [ 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:182 3.7s0 3:ea2a6e9a:::smithi10238019-22:head [copy-from ver 8] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:06.273 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 75.592493176999994, 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.030026536, 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 182 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677855+0000", 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677857+0000", 2024-09-17T17:31:06.275 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2839999999999999e-06 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677863+0000", 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8850000000000004e-06 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677875+0000", 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2490999999999999e-05 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.276 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677882+0000", 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.652e-06 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677943+0000", 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1327000000000004e-05 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.277 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678017+0000", 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3440000000000002e-05 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.703307+0000", 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.025289977000000002 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704581+0000", 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0012741790000000001 2024-09-17T17:31:06.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707797+0000", 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.003216642 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.707882+0000", 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.4659000000000004e-05 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:183 3.7s0 3:ea2a6e9a:::smithi10238019-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 75.592454465000003, 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026638608000000001, 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 183 2024-09-17T17:31:06.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.281 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677894+0000", 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2099999999999998e-07 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677896+0000", 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.804e-06 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677902+0000", 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5149999999999997e-06 2024-09-17T17:31:06.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.677905+0000", 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.5209999999999998e-06 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678152+0000", 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000246536 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.283 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.678180+0000", 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8605000000000001e-05 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704308+0000", 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026127128999999999 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704406+0000", 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.8079999999999996e-05 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:50.704533+0000", 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012709700000000001 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 43/32 ECSubWriteReply(tid=93, last_complete=43'8, committed=1, applied=1))", 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 71.563220228999995, 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.427260368, 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:31:06.286 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707128+0000", 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707129+0000", 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9599999999999999e-07 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.287 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:54.707133+0000", 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0629999999999999e-06 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134286+0000", 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.427152792 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134295+0000", 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.9500000000000007e-06 2024-09-17T17:31:06.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134356+0000", 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.1439000000000006e-05 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:29:55.134389+0000", 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2428000000000002e-05 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:489 3.1s0 3:87ca9cdd:::smithi10238019-41:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.900834531000001, 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01402203, 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.290 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 489 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369514+0000", 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369517+0000", 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.8480000000000001e-06 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369527+0000", 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0037000000000001e-05 2024-09-17T17:31:06.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369548+0000", 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.0653e-05 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369556+0000", 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.5909999999999996e-06 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.293 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369630+0000", 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.3579999999999997e-05 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "op must wait for map", 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369636+0000", 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6900000000000003e-06 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.373208+0000", 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0035712719999999999 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.374240+0000", 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0010317569999999999 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.375226+0000", 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00098623599999999993 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.380028+0000", 2024-09-17T17:31:06.295 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0048024870000000002 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.383466+0000", 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0034380299999999999 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.383536+0000", 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.9849000000000002e-05 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:490 3.1s0 3:87ca9cdd:::smithi10238019-41:head [rollback 9] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.900770727000001, 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016574469000000001, 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 490 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:31:06.298 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369578+0000", 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5999999999999999e-07 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369581+0000", 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5000000000000002e-06 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.299 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369586+0000", 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7570000000000004e-06 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369589+0000", 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.7340000000000002e-06 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369733+0000", 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014407800000000001 2024-09-17T17:31:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369741+0000", 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.8310000000000001e-06 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.376615+0000", 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0068736020000000004 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.376737+0000", 2024-09-17T17:31:06.301 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00012183199999999999 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.377593+0000", 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00085658399999999997 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.381579+0000", 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0039858879999999999 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.302 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386071+0000", 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0044917280000000004 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386152+0000", 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1475000000000002e-05 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.303 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:491 3.1s0 3:87ca9cdd:::smithi10238019-41:head [append 0~0] snapc c=[9] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.900731573999998, 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016871317, 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 491 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369617+0000", 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.8799999999999998e-07 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369623+0000", 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0730000000000004e-06 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.306 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369629+0000", 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.0310000000000004e-06 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369633+0000", 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9899999999999999e-06 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369901+0000", 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00026797300000000002 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.369915+0000", 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4143e-05 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.378961+0000", 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0090466739999999993 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.308 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379126+0000", 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016480800000000001 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.379682+0000", 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00055593299999999999 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.381583+0000", 2024-09-17T17:31:06.309 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001901123 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386406+0000", 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0048232199999999996 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.386488+0000", 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1960999999999997e-05 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:492 3.bs0 3:d4565dea:::smithi10238019-24:head [delete] snapc c=[9] ondisk+write+known_if_redirected+full_force+supports_pool_eio e57)", 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 57.846630988999998, 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.23375553299999999, 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 492 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423718+0000", 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423720+0000", 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.4530000000000001e-06 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423731+0000", 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.1127e-05 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423762+0000", 2024-09-17T17:31:06.313 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1028999999999997e-05 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423777+0000", 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4357e-05 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.423949+0000", 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017206400000000001 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.314 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.424109+0000", 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000160703 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.424863+0000", 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00075321999999999997 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.425916+0000", 2024-09-17T17:31:06.315 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001053888 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.657390+0000", 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.231473764 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:08.657473+0000", 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2928000000000003e-05 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.316 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:582 3.6s0 3:672dafef:::smithi10238019-1:head [copy-from ver 5] snapc 11=[11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 49.908394268000002, 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.019590435999999999, 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.317 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 582 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361954+0000", 2024-09-17T17:31:06.318 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361956+0000", 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4789999999999999e-06 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361969+0000", 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.2979999999999999e-05 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.319 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361983+0000", 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.3917000000000001e-05 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.361992+0000", 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.8850000000000005e-06 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364412+0000", 2024-09-17T17:31:06.320 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002420082 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364470+0000", 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.8588999999999998e-05 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.379219+0000", 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.014748225 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.321 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380383+0000", 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0011644050000000001 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.381488+0000", 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.001105225 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.381545+0000", 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.6648999999999999e-05 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.322 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:583 3.6s0 3:672dafef:::smithi10238019-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 49.908320367999998, 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.018119641999999998, 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.323 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 583 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362028+0000", 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362029+0000", 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.5799999999999999e-07 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362032+0000", 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.2729999999999998e-06 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.325 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362040+0000", 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.5650000000000004e-06 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.362044+0000", 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5549999999999996e-06 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364703+0000", 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0026589909999999999 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.326 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.364754+0000", 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0785999999999998e-05 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380012+0000", 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015258315 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380076+0000", 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.368e-05 2024-09-17T17:31:06.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:16.380148+0000", 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1919e-05 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 68/67 ECSubWriteReply(tid=199, last_complete=68'121, committed=1, applied=1))", 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.766358209000003, 2024-09-17T17:31:06.328 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.188788604, 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503990+0000", 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503991+0000", 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7700000000000003e-07 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.503996+0000", 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4039999999999997e-06 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.330 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692049+0000", 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.188053784 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692058+0000", 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.1480000000000006e-06 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692105+0000", 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.6878000000000001e-05 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.692779+0000", 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00067451299999999996 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/67 ECSubWriteReply(tid=202, last_complete=68'119, committed=1, applied=1))", 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 46.570477187000002, 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.16860020100000001, 2024-09-17T17:31:06.332 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699871+0000", 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699872+0000", 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.2699999999999998e-07 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.699877+0000", 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7090000000000001e-06 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868220+0000", 2024-09-17T17:31:06.334 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.168342782 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868242+0000", 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.1985999999999999e-05 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868400+0000", 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000158331 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.335 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:19.868472+0000", 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.1565999999999999e-05 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 70/67 ECSubWriteReply(tid=208, last_complete=70'125, committed=1, applied=1))", 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.906347527999998, 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.158690363, 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364001+0000", 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.337 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364002+0000", 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.1199999999999998e-07 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.364005+0000", 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.117e-06 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522493+0000", 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.15848867899999999 2024-09-17T17:31:06.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522508+0000", 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.4419999999999999e-05 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522668+0000", 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00016041699999999999 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.339 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.522691+0000", 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3317999999999999e-05 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 70/67 ECSubWriteReply(tid=208, last_complete=70'125, committed=1, applied=1))", 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 44.901986542000003, 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.71001705199999998, 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368362+0000", 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368363+0000", 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.7199999999999997e-07 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:21.368367+0000", 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3680000000000004e-06 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078255+0000", 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.70988762599999999 2024-09-17T17:31:06.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078263+0000", 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7430000000000002e-06 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078349+0000", 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 8.6163999999999997e-05 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.343 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:22.078379+0000", 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.0278999999999999e-05 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 75/74 ECSubWriteReply(tid=239, last_complete=75'130, committed=1, applied=1))", 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 39.893985119, 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.20952685800000001, 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.345 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376364+0000", 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.68e-07 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.376368+0000", 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6330000000000002e-06 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585670+0000", 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.20930257699999999 2024-09-17T17:31:06.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585686+0000", 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.5676e-05 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585835+0000", 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00014933399999999999 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:26.585890+0000", 2024-09-17T17:31:06.347 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4969999999999997e-05 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:796 3.16s0 3:6f166faa:::smithi10238019-38:head [copy-from ver 25] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 35.909376704000003, 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.020945305000000001, 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.348 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 796 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.349 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360972+0000", 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.01e-07 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360975+0000", 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5620000000000002e-06 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360981+0000", 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.4989999999999999e-06 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360988+0000", 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.3369999999999996e-06 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361038+0000", 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.0141e-05 2024-09-17T17:31:06.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361130+0000", 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1666e-05 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.374988+0000", 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.01385871 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.352 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.377231+0000", 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022428610000000001 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.381852+0000", 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.004620613 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.381917+0000", 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.5514999999999999e-05 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.353 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:797 3.16s0 3:6f166faa:::smithi10238019-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 35.909349632000001, 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.016029564999999999, 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.354 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 797 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.360999+0000", 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.4700000000000002e-07 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361002+0000", 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.3929999999999998e-06 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.356 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361008+0000", 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6470000000000001e-06 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361014+0000", 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.4650000000000002e-06 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361241+0000", 2024-09-17T17:31:06.357 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000227035 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.361277+0000", 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.6492000000000002e-05 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.376753+0000", 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.015475401999999999 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.376852+0000", 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.9086999999999995e-05 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:30.377029+0000", 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00017659699999999999 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:841 3.16s0 3:6f166faa:::smithi10238019-38:head [copy-from ver 27] snapc 1c=[1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.339062+0000", 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.931286153999999, 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.053373410000000003, 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 841 2024-09-17T17:31:06.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339062+0000", 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339062+0000", 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339065+0000", 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339075+0000", 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0540000000000001e-05 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339095+0000", 2024-09-17T17:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.9737000000000001e-05 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339105+0000", 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.0526000000000001e-05 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339197+0000", 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.1257999999999999e-05 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "op must wait for map", 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339206+0000", 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 9.0599999999999997e-06 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.341465+0000", 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.002258917 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.341601+0000", 2024-09-17T17:31:06.364 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00013603199999999999 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.363580+0000", 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.02197928 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.366338+0000", 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0027583389999999998 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.365 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.392376+0000", 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026037919999999999 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.392436+0000", 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.9599000000000003e-05 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.366 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:842 3.16s0 3:6f166faa:::smithi10238019-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.931221918999999, 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.026888496000000001, 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 842 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339127+0000", 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.3700000000000001e-07 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339130+0000", 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 2.5129999999999999e-06 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339137+0000", 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.0870000000000001e-06 2024-09-17T17:31:06.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339141+0000", 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0230000000000003e-06 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339270+0000", 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000129326 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:06.370 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.339274+0000", 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.9700000000000001e-06 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.342500+0000", 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0032258740000000001 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.342566+0000", 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6150999999999997e-05 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.365784+0000", 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.023217987999999998 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.365901+0000", 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000116741 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.366015+0000", 2024-09-17T17:31:06.372 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000114386 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:847 3.16s0 3:6d14b604:::smithi10238019-45:head [setxattr _header (57) in=7b] snapc 1c=[1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.400677+0000", 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.869671234999998, 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.12075385900000001, 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:06.373 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 847 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400677+0000", 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.374 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400677+0000", 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400678+0000", 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 5.2399999999999998e-07 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400682+0000", 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.0509999999999998e-06 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.375 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400687+0000", 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.7999999999999998e-06 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400690+0000", 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3689999999999998e-06 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402064+0000", 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0013741160000000001 2024-09-17T17:31:06.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402168+0000", 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00010328 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402694+0000", 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00052647200000000005 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.377 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.403384+0000", 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00068960500000000004 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "commit_sent", 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521353+0000", 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.11796981600000001 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521431+0000", 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 7.7825999999999997e-05 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "description": "osd_op(client.4566.0:848 3.16s0 3:6d14b604:::smithi10238019-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "initiated_at": "2024-09-17T17:30:36.400695+0000", 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 29.869653387, 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.121926795, 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "continuous": false, 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type_data": { 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flag_point": "started", 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "client_info": { 2024-09-17T17:31:06.379 INFO:teuthology.orchestra.run.smithi026.stdout: "client": "client.4566", 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tid": 848 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "events": [ 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "initiated", 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400695+0000", 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "header_read", 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400695+0000", 2024-09-17T17:31:06.380 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "throttled", 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400696+0000", 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.3099999999999999e-07 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "all_read", 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400697+0000", 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 1.7039999999999999e-06 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.381 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "dispatched", 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400702+0000", 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 4.5310000000000003e-06 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.400705+0000", 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 3.1750000000000001e-06 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.402985+0000", 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.0022796460000000002 2024-09-17T17:31:06.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.403051+0000", 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 6.6756999999999998e-05 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "reached_pg", 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521810+0000", 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.118758407 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "started", 2024-09-17T17:31:06.383 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.521912+0000", 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.000101847 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: "event": "done", 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: "time": "2024-09-17T17:30:36.522622+0000", 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: "duration": 0.00071039699999999998 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-17T17:31:06.385 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-17T17:31:06.385 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-17T17:31:06.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T17:31:06.506 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:31:06.506 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:06.516 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T17:31:06.625 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:31:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:31:06.625 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:06.626 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T17:31:06.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 75/74 ECSubWrite(tid=235, reqid=osd.13.0:235, at_version=75'127, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.379513725000002, 2024-09-17T17:31:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069013160000000002, 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.360999+0000", 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1200000000000003e-07 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361014+0000", 2024-09-17T17:31:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4908e-05 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361042+0000", 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8481000000000001e-05 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361050+0000", 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7109999999999995e-06 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361102+0000", 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1959999999999997e-05 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.361112+0000", 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0345e-05 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.367865+0000", 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067524300000000002 2024-09-17T17:31:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.367900+0000", 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4968999999999997e-05 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 75/74 ECSubWrite(tid=236, reqid=osd.13.0:236, at_version=75'128, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:31:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.377534658999998, 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077947959999999997, 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362978+0000", 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.362994+0000", 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5650000000000001e-05 2024-09-17T17:31:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363018+0000", 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4300000000000001e-05 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363026+0000", 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4540000000000001e-06 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363078+0000", 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2086999999999998e-05 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363088+0000", 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.005e-05 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370737+0000", 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076488509999999999 2024-09-17T17:31:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.370772+0000", 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5858999999999997e-05 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.16s3 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:31:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.376676473000003, 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095484720000000006, 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363836+0000", 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2799999999999996e-07 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363852+0000", 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515e-05 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363883+0000", 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1594999999999998e-05 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.363893+0000", 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1509999999999993e-06 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365558+0000", 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001665281 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.365563+0000", 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3410000000000002e-06 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373353+0000", 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077893490000000001 2024-09-17T17:31:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.373384+0000", 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1711999999999997e-05 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=106, reqid=client.4566.0:786, at_version=79'58, trim_to=0'0, roll_forward_to=78'54))", 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.379598801, 2024-09-17T17:31:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0061729920000000004, 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360914+0000", 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360915+0000", 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1769999999999999e-06 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361086+0000", 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017148 2024-09-17T17:31:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361114+0000", 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7773999999999999e-05 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361124+0000", 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0380999999999999e-05 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361197+0000", 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3140999999999996e-05 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361218+0000", 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0197e-05 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367041+0000", 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005823249 2024-09-17T17:31:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367086+0000", 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5593000000000002e-05 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.13s5 79/74 ECSubWrite(tid=107, reqid=client.4566.0:787, at_version=79'59, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:31:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.377629149000001, 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0070290120000000003, 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362883+0000", 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362884+0000", 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.037e-06 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.362968+0000", 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3614000000000003e-05 2024-09-17T17:31:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363001+0000", 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3385e-05 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363013+0000", 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1761e-05 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363106+0000", 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2560999999999997e-05 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.363117+0000", 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1457999999999999e-05 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369881+0000", 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006764417 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.369912+0000", 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0778999999999998e-05 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 91/76 ECSubWrite(tid=201, reqid=osd.12.0:201, at_version=91'65, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.386760515999999, 2024-09-17T17:31:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060328960000000003, 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:31:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353752+0000", 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4900000000000001e-07 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353782+0000", 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9933999999999998e-05 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353905+0000", 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012266299999999999 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353914+0000", 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2050000000000008e-06 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.354626+0000", 2024-09-17T17:31:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071210800000000003 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.354639+0000", 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2594e-05 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.359660+0000", 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050218559999999999 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.359785+0000", 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000124187 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=202, reqid=osd.12.0:202, at_version=91'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.386727257, 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0082767129999999998, 2024-09-17T17:31:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353785+0000", 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5100000000000001e-07 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.353809+0000", 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4088999999999999e-05 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356309+0000", 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024992579999999999 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356319+0000", 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7629999999999995e-06 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356466+0000", 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147269 2024-09-17T17:31:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356477+0000", 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1569000000000001e-05 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.362024+0000", 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055471019999999999 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.362062+0000", 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7311999999999998e-05 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=203, reqid=client.4566.0:889, at_version=91'75, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.355806+0000", 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.384706056999999, 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012494346, 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.355806+0000", 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.355806+0000", 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.355807+0000", 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7599999999999997e-07 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.356290+0000", 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048311300000000002 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357481+0000", 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011908070000000001 2024-09-17T17:31:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357491+0000", 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7659999999999993e-06 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.358479+0000", 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00098815800000000005 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.358489+0000", 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0084999999999999e-05 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368256+0000", 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0097670750000000001 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368301+0000", 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4666000000000002e-05 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=204, reqid=client.4566.0:890, at_version=91'76, trim_to=0'0, roll_forward_to=89'71))", 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.357411+0000", 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.383100854999999, 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01092184, 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357411+0000", 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357411+0000", 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357413+0000", 2024-09-17T17:31:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0860000000000001e-06 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.357458+0000", 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5321999999999997e-05 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.360367+0000", 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029088289999999999 2024-09-17T17:31:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.360376+0000", 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6160000000000008e-06 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.361787+0000", 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014109420000000001 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.361806+0000", 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8380999999999999e-05 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368305+0000", 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064998740000000001 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368333+0000", 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.779e-05 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/75 ECSubWrite(tid=206, reqid=client.4566.0:890, at_version=0'0, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.371541607000001, 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010717031, 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368971+0000", 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.368984+0000", 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2493e-05 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373159+0000", 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041748449999999999 2024-09-17T17:31:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373167+0000", 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2700000000000004e-06 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373249+0000", 2024-09-17T17:31:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1790000000000001e-05 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373260+0000", 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.115e-05 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.379654+0000", 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0063941529999999996 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.379688+0000", 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3578999999999998e-05 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s3 91/74 ECSubWrite(tid=307, reqid=client.4566.0:903, at_version=0'0, trim_to=0'0, roll_forward_to=91'133))", 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.372644+0000", 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.367867960000002, 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078174990000000003, 2024-09-17T17:31:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372644+0000", 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372644+0000", 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372645+0000", 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2399999999999997e-07 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372657+0000", 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.182e-05 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372675+0000", 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.77e-05 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372709+0000", 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4307999999999997e-05 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372793+0000", 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4443000000000004e-05 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372937+0000", 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014369299999999999 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380433+0000", 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.007495832 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380462+0000", 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8979000000000002e-05 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.0s3 91/75 ECSubRead(tid=210, to_read={3:00c2d3b1:::smithi10238019-29:head=204800,258048,0}, subchunks={3:00c2d3b1:::smithi10238019-29:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.366671094000001, 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.94432460600000001, 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373841+0000", 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373845+0000", 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9469999999999996e-06 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.317764+0000", 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.94391877700000004 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.317783+0000", 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9037000000000001e-05 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.317920+0000", 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000137059 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.318166+0000", 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245541 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 92/75 ECSubWrite(tid=211, reqid=osd.12.0:211, at_version=92'78, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.386976299000001, 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0067417500000000003, 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353536+0000", 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3300000000000001e-07 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353575+0000", 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8878999999999998e-05 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353655+0000", 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9399000000000003e-05 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353667+0000", 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2383e-05 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353745+0000", 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7644000000000004e-05 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353757+0000", 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2074e-05 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.360229+0000", 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064723640000000004 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.360278+0000", 2024-09-17T17:31:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8773999999999998e-05 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s3 92/75 ECSubWrite(tid=212, reqid=osd.12.0:212, at_version=92'80, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.386546706000001, 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0071408280000000001, 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.353966+0000", 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5700000000000002e-07 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354034+0000", 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7350999999999999e-05 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354071+0000", 2024-09-17T17:31:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6807000000000002e-05 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354079+0000", 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4270000000000008e-06 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354185+0000", 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105991 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354195+0000", 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8460000000000003e-06 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361063+0000", 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068683379999999999 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361106+0000", 2024-09-17T17:31:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3310999999999999e-05 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s6 92/74 ECSubWrite(tid=316, reqid=osd.13.0:316, at_version=92'34, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.355682+0000", 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.384829832000001, 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060278060000000001, 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355682+0000", 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355682+0000", 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355686+0000", 2024-09-17T17:31:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.461e-06 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355713+0000", 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.707e-05 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355795+0000", 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1611000000000002e-05 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355803+0000", 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1440000000000003e-06 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355897+0000", 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4637999999999998e-05 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355910+0000", 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2306e-05 2024-09-17T17:31:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361664+0000", 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057543170000000001 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.361710+0000", 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6258999999999999e-05 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.3s5 92/74 ECSubWrite(tid=128, reqid=osd.10.0:128, at_version=92'78, trim_to=0'0, roll_forward_to=85'76))", 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.384546599, 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0098776119999999992, 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.355966+0000", 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.34e-07 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356001+0000", 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5017e-05 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356088+0000", 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6717000000000003e-05 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356096+0000", 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.208e-06 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356167+0000", 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1194999999999999e-05 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356177+0000", 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7089999999999997e-06 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.365807+0000", 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0096299200000000001 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.365843+0000", 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6511999999999999e-05 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 92/76 ECSubWrite(tid=213, reqid=osd.12.0:213, at_version=92'67, trim_to=0'0, roll_forward_to=91'65))", 2024-09-17T17:31:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.356380+0000", 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.384132427000001, 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006989337, 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356380+0000", 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356380+0000", 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356381+0000", 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6300000000000005e-07 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356400+0000", 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9814999999999999e-05 2024-09-17T17:31:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356442+0000", 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2042000000000002e-05 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356452+0000", 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4590000000000004e-06 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356511+0000", 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9218000000000001e-05 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356524+0000", 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3086e-05 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363325+0000", 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0068006719999999998 2024-09-17T17:31:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363369+0000", 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4382000000000003e-05 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s6 96/74 ECSubWrite(tid=330, reqid=client.4566.0:927, at_version=96'46, trim_to=0'0, roll_forward_to=96'42))", 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.360632+0000", 2024-09-17T17:31:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.379879875, 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080918599999999993, 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360632+0000", 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360632+0000", 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360633+0000", 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0049999999999999e-06 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360781+0000", 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014790199999999999 2024-09-17T17:31:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360806+0000", 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4391000000000001e-05 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360813+0000", 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1779999999999997e-06 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360874+0000", 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1099000000000006e-05 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.360885+0000", 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1051e-05 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368683+0000", 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077983890000000002 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368724+0000", 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0844999999999999e-05 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s6 96/74 ECSubWrite(tid=331, reqid=client.4566.0:928, at_version=96'47, trim_to=0'0, roll_forward_to=96'42))", 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.362531+0000", 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.377981316, 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0062230469999999998, 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362531+0000", 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362531+0000", 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362532+0000", 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0800000000000004e-07 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362551+0000", 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9377000000000001e-05 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362586+0000", 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5414999999999999e-05 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362595+0000", 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0269999999999995e-06 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.363730+0000", 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011350519999999999 2024-09-17T17:31:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.363738+0000", 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8589999999999994e-06 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368730+0000", 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049914970000000001 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.368754+0000", 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4111999999999999e-05 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 100/76 ECSubWrite(tid=237, reqid=client.4566.0:969, at_version=100'75, trim_to=0'0, roll_forward_to=100'71))", 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.967972792999999, 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075688450000000003, 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.772540+0000", 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2699999999999999e-07 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774069+0000", 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015291079999999999 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774101+0000", 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1423000000000001e-05 2024-09-17T17:31:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774115+0000", 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.488e-05 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774190+0000", 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4799e-05 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.774198+0000", 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8110000000000003e-06 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.780079+0000", 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0058809120000000003 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.780108+0000", 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9385000000000001e-05 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:06.842 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:06.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T17:31:06.950 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:31:06.951 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:06.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T17:31:07.065 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:31:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:31:07.066 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:07.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T17:31:07.182 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 97.599907177000006, 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.27275196200000001, 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580025+0000", 2024-09-17T17:31:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580026+0000", 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.359e-06 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580034+0000", 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580048+0000", 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3974e-05 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580058+0000", 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0064999999999999e-05 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580111+0000", 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3248000000000002e-05 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.580119+0000", 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1270000000000003e-06 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847220+0000", 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26710079599999997 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847357+0000", 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013739500000000001 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.847505+0000", 2024-09-17T17:31:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147188 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.850265+0000", 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002760396 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852701+0000", 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024356059999999999 2024-09-17T17:31:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852744+0000", 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3095e-05 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.852777+0000", 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2784000000000003e-05 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 97.192418888999995, 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010965536999999999, 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 48 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987513+0000", 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5899999999999996e-07 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987518+0000", 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0429999999999998e-06 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987523+0000", 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3499999999999999e-06 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987526+0000", 2024-09-17T17:31:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5159999999999999e-06 2024-09-17T17:31:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991339+0000", 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038128670000000002 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991396+0000", 2024-09-17T17:31:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7219000000000002e-05 2024-09-17T17:31:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991498+0000", 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010142 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993114+0000", 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016166900000000001 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998400+0000", 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052850600000000003 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998438+0000", 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8272000000000002e-05 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998478+0000", 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0540999999999997e-05 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 97.192400539000005, 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011028196000000001, 2024-09-17T17:31:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 49 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987531+0000", 2024-09-17T17:31:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8000000000000002e-07 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987534+0000", 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4399999999999999e-06 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987538+0000", 2024-09-17T17:31:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9369999999999997e-06 2024-09-17T17:31:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987541+0000", 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0350000000000002e-06 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991920+0000", 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043794000000000003 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.991972+0000", 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1412000000000002e-05 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992057+0000", 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4917e-05 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993157+0000", 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011003810000000001 2024-09-17T17:31:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998485+0000", 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053282989999999999 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998517+0000", 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2019e-05 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998559+0000", 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2076000000000002e-05 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 97.19237905, 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011080325, 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 50 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987553+0000", 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-07 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987605+0000", 2024-09-17T17:31:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2114000000000003e-05 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987631+0000", 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5766999999999999e-05 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987646+0000", 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5515999999999998e-05 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992371+0000", 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047247360000000002 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992433+0000", 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1451000000000005e-05 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.992522+0000", 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9621000000000001e-05 2024-09-17T17:31:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994510+0000", 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001988227 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998564+0000", 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040533710000000001 2024-09-17T17:31:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998595+0000", 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0843000000000003e-05 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998633+0000", 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8321000000000001e-05 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 97.192267025000007, 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011043702000000001, 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 51 2024-09-17T17:31:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:31:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:31:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987665+0000", 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987669+0000", 2024-09-17T17:31:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7189999999999999e-06 2024-09-17T17:31:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987674+0000", 2024-09-17T17:31:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1780000000000002e-06 2024-09-17T17:31:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987678+0000", 2024-09-17T17:31:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5999999999999998e-06 2024-09-17T17:31:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993192+0000", 2024-09-17T17:31:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055137620000000002 2024-09-17T17:31:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993262+0000", 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0208000000000003e-05 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.993349+0000", 2024-09-17T17:31:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7362999999999997e-05 2024-09-17T17:31:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994549+0000", 2024-09-17T17:31:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011994110000000001 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998640+0000", 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040912409999999998 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998669+0000", 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9209000000000001e-05 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998708+0000", 2024-09-17T17:31:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9419000000000003e-05 2024-09-17T17:31:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:31:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 97.192246935, 2024-09-17T17:31:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011098023, 2024-09-17T17:31:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 52 2024-09-17T17:31:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987685+0000", 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2000000000000001e-07 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987688+0000", 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4729999999999999e-06 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987693+0000", 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5010000000000004e-06 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.987698+0000", 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1499999999999998e-06 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994130+0000", 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064314890000000003 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994185+0000", 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5769000000000001e-05 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994272+0000", 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6638000000000006e-05 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.994990+0000", 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000717463 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998715+0000", 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003725 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998743+0000", 2024-09-17T17:31:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8830999999999999e-05 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:29.998783+0000", 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9388999999999998e-05 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 96.882532440999995, 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011397086000000001, 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 76 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297399+0000", 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297400+0000", 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7100000000000002e-07 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297403+0000", 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5920000000000001e-06 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297410+0000", 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1489999999999997e-06 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.297417+0000", 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-06 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306131+0000", 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087138279999999999 2024-09-17T17:31:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306185+0000", 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3783999999999999e-05 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306278+0000", 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3587999999999999e-05 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.306994+0000", 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071571099999999995 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308729+0000", 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001735489 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308769+0000", 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9270000000000002e-05 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.308796+0000", 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7603999999999999e-05 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4211.0:124 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:31:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 96.640470888999999, 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011177351, 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4211", 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/3387577180", 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 124 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:31:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539461+0000", 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539467+0000", 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2089999999999996e-06 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539473+0000", 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4690000000000002e-06 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.539479+0000", 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-17T17:31:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545059+0000", 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055806400000000004 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545104+0000", 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5040999999999998e-05 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 0", 2024-09-17T17:31:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545172+0000", 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7249999999999995e-05 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.545874+0000", 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070271799999999996 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550512+0000", 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046379530000000002 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550597+0000", 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025999999999995e-05 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:30.550638+0000", 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0771e-05 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.as2 37/32 ECSubRead(tid=27, to_read={3:5ee91470:::smithi10238019-6:head=20480,4096,0}, subchunks={3:5ee91470:::smithi10238019-6:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 78.186905136999997, 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.671263006, 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993027+0000", 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993028+0000", 2024-09-17T17:31:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0219999999999999e-06 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:48.993040+0000", 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1968e-05 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664125+0000", 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.67108543799999998 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664134+0000", 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8640000000000002e-06 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664204+0000", 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9733999999999993e-05 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:49.664290+0000", 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.598e-05 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 43/35 ECSubWrite(tid=54, reqid=osd.11.0:54, at_version=43'10, trim_to=0'0, roll_forward_to=42'8))", 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 72.478413184999994, 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.282394969, 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701519+0000", 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6e-07 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.701543+0000", 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4530000000000001e-05 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983000+0000", 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.28145619700000002 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983008+0000", 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6510000000000007e-06 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983087+0000", 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8623999999999998e-05 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983093+0000", 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6599999999999998e-06 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983867+0000", 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000773962 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.983914+0000", 2024-09-17T17:31:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6084999999999997e-05 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 43/35 ECSubWrite(tid=62, reqid=client.4566.0:316, at_version=43'14, trim_to=0'0, roll_forward_to=43'10))", 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 72.188966234000006, 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.039163349, 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990965+0000", 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990966+0000", 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3e-07 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.990998+0000", 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1909000000000002e-05 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029506+0000", 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.038507953999999997 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029512+0000", 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3419999999999996e-06 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029561+0000", 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8930000000000001e-05 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.029568+0000", 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2480000000000001e-06 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.030089+0000", 2024-09-17T17:31:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00052149900000000003 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:55.030129+0000", 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9737000000000003e-05 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.fs4 56/35 ECSubWrite(tid=133, reqid=client.4566.0:486, at_version=56'48, trim_to=0'0, roll_forward_to=56'45))", 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.799500270000003, 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31954210500000002, 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380431+0000", 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380432+0000", 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8500000000000001e-07 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.380446+0000", 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4214e-05 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699334+0000", 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31888741599999998 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699350+0000", 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5913000000000002e-05 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699521+0000", 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000171244 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699530+0000", 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8079999999999994e-06 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699939+0000", 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040906299999999998 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.699974+0000", 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4761999999999997e-05 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.18s0 68/67 ECSubWriteReply(tid=153, last_complete=68'25, committed=1, applied=1))", 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.716003082, 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.73874762000000005, 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463929+0000", 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0400000000000002e-07 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.463933+0000", 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9689999999999996e-06 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202459+0000", 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.738526142 2024-09-17T17:31:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202472+0000", 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2989999999999999e-05 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202624+0000", 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015171499999999999 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.202676+0000", 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.24e-05 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s1 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.703941522000001, 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.37910365400000001, 2024-09-17T17:31:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475990+0000", 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475991+0000", 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476005+0000", 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.449e-05 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854381+0000", 2024-09-17T17:31:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.37837595699999998 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854387+0000", 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4689999999999996e-06 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854441+0000", 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4593999999999999e-05 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.854449+0000", 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0029999999999994e-06 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.855065+0000", 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061543700000000004 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.855094+0000", 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9193999999999999e-05 2024-09-17T17:31:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:799 3.18s0 3:1f25649f:::smithi10238019-40:head [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339002+0000", 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.840930010000001, 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012582919, 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 799 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339002+0000", 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339002+0000", 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339005+0000", 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3139999999999999e-06 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339032+0000", 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6529999999999998e-05 2024-09-17T17:31:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339056+0000", 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.421e-05 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339069+0000", 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339161+0000", 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2238999999999996e-05 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339171+0000", 2024-09-17T17:31:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9089999999999995e-06 2024-09-17T17:31:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348303+0000", 2024-09-17T17:31:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0091322829999999997 2024-09-17T17:31:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348416+0000", 2024-09-17T17:31:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112928 2024-09-17T17:31:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.351585+0000", 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031685929999999999 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:800 3.18s0 3:1f25649f:::smithi10238019-40:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T17:31:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339103+0000", 2024-09-17T17:31:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.840828916, 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013185367999999999, 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 800 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339103+0000", 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339103+0000", 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339104+0000", 2024-09-17T17:31:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1100000000000005e-07 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339131+0000", 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7180999999999999e-05 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339211+0000", 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9802000000000006e-05 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339221+0000", 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0277e-05 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339434+0000", 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0002131 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339442+0000", 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6620000000000005e-06 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348818+0000", 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093763130000000007 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.348904+0000", 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6069000000000006e-05 2024-09-17T17:31:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.352288+0000", 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003384153 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:801 3.18s0 3:1f25649f:::smithi10238019-40:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.339263+0000", 2024-09-17T17:31:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.840668915999998, 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.98368640699999998, 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 801 2024-09-17T17:31:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339263+0000", 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339263+0000", 2024-09-17T17:31:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339264+0000", 2024-09-17T17:31:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1850000000000001e-06 2024-09-17T17:31:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339271+0000", 2024-09-17T17:31:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7299999999999999e-06 2024-09-17T17:31:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339281+0000", 2024-09-17T17:31:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0674e-05 2024-09-17T17:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339290+0000", 2024-09-17T17:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3450000000000006e-06 2024-09-17T17:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339467+0000", 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000177636 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.339473+0000", 2024-09-17T17:31:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7409999999999998e-06 2024-09-17T17:31:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.349129+0000", 2024-09-17T17:31:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.009656296 2024-09-17T17:31:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.349185+0000", 2024-09-17T17:31:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6057999999999998e-05 2024-09-17T17:31:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322949+0000", 2024-09-17T17:31:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.97376374200000004 2024-09-17T17:31:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.18s0 80/75 ECSubReadReply(tid=169, attrs_read=0))", 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.829147177999999, 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.97217706000000004, 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350785+0000", 2024-09-17T17:31:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8000000000000001e-07 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:31.350860+0000", 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5136999999999996e-05 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322452+0000", 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.97159191899999997 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322462+0000", 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9559999999999992e-06 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322532+0000", 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0222999999999995e-05 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:32.322962+0000", 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00042944499999999998 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:964 3.18s0 3:1f25649f:::smithi10238019-40:head [copy-from ver 60] snapc 22=[22,21,20,18,17,10] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:51.753402+0000", 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.426529735000001, 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.042342484, 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 964 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753402+0000", 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753402+0000", 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753414+0000", 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2306999999999999e-05 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753449+0000", 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5151999999999998e-05 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753481+0000", 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1702999999999998e-05 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753489+0000", 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.729e-06 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753555+0000", 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6277000000000003e-05 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753640+0000", 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5048000000000003e-05 2024-09-17T17:31:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.787721+0000", 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.034080810000000003 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.791433+0000", 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037123429999999999 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.795661+0000", 2024-09-17T17:31:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042272840000000004 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.795744+0000", 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3831000000000004e-05 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:965 3.18s0 3:1f25649f:::smithi10238019-40:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:51.753508+0000", 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.426423368, 2024-09-17T17:31:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.037400758999999999, 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 965 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753508+0000", 2024-09-17T17:31:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753508+0000", 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753509+0000", 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8100000000000002e-07 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753515+0000", 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3910000000000003e-06 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753525+0000", 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4320000000000005e-06 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753529+0000", 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6430000000000003e-06 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753787+0000", 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025746599999999998 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.753845+0000", 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7667999999999997e-05 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.790726+0000", 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.036881635000000003 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.790817+0000", 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1205000000000005e-05 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:51.790909+0000", 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1638000000000006e-05 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.322 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:07.324 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-17T17:31:07.431 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:31:07.431 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:07.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-17T17:31:07.547 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:31:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:31:07.548 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:07.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-17T17:31:07.623 INFO:teuthology.orchestra.run.smithi007.stdout:get_pg_num_history pg_num_history pg_num_history(e110 pg_nums {1={23=1},2={25=8},3={32=16,67=26}} deleted_pools ) 2024-09-17T17:31:07.623 INFO:teuthology.orchestra.run.smithi007.stdout:Importing pgid 3.bs4 2024-09-17T17:31:07.623 INFO:teuthology.orchestra.run.smithi007.stdout:Write 4#3:d4565dea:::smithi10238019-24:head# 2024-09-17T17:31:07.623 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 1e={} 2024-09-17T17:31:07.625 INFO:teuthology.orchestra.run.smithi007.stdout:write_pg epoch 109 info 3.bs4( v 96'22 (0'0,96'22] local-lis/les=95/96 n=1 ec=32/32 lis/c=95/95 les/c/f=96/96/0 sis=95) 2024-09-17T17:31:07.625 INFO:teuthology.orchestra.run.smithi007.stdout:Import successful 2024-09-17T17:31:07.664 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.664 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:218 3.3s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (56) in=7b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.940208916000003, 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65573700899999998, 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 218 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721615+0000", 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721616+0000", 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4e-07 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721619+0000", 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2030000000000001e-06 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721626+0000", 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8510000000000001e-06 2024-09-17T17:31:07.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721631+0000", 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4539999999999998e-06 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722624+0000", 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00099297500000000011 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722675+0000", 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0627000000000002e-05 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.722943+0000", 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026790700000000001 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723463+0000", 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000520003 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377270+0000", 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65380684099999997 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377352+0000", 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2307999999999999e-05 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.670 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:219 3.3s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.940182691999993, 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65649164999999998, 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 219 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721641+0000", 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721642+0000", 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0100000000000005e-07 2024-09-17T17:31:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721644+0000", 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4719999999999998e-06 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721651+0000", 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.781e-06 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.721656+0000", 2024-09-17T17:31:07.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4809999999999997e-06 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723147+0000", 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001490587 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:50.723197+0000", 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.003e-05 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377653+0000", 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65445650300000002 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.377716+0000", 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2223000000000004e-05 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:51.378133+0000", 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041707199999999999 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:440 3.3s0 3:c80ea1ca:::smithi10238019-27:head [append 0~81920] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.257449170000001, 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19940718800000001, 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 440 2024-09-17T17:31:07.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404375+0000", 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8299999999999996e-07 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404418+0000", 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2608000000000003e-05 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404437+0000", 2024-09-17T17:31:07.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8810000000000001e-05 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.404442+0000", 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.327e-06 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406246+0000", 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018034769999999999 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406352+0000", 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106269 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.406784+0000", 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00043238800000000002 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.408111+0000", 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001326502 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603719+0000", 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.195608474 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603782+0000", 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2550000000000003e-05 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/35 ECSubWriteReply(tid=48, last_complete=52'36, committed=1, applied=1))", 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.254054031999999, 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19602166600000001, 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407770+0000", 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407771+0000", 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1100000000000004e-07 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.407775+0000", 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5510000000000001e-06 2024-09-17T17:31:07.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603542+0000", 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19576712299999999 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603557+0000", 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4229999999999999e-05 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603658+0000", 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010186100000000001 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.603791+0000", 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013299000000000001 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.as3 62/32 ECSubWrite(tid=104, reqid=client.4566.0:558, at_version=62'84, trim_to=0'0, roll_forward_to=62'79))", 2024-09-17T17:31:07.685 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.159888998, 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95386532199999996, 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501935+0000", 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501936+0000", 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0109999999999999e-06 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:13.501948+0000", 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2544000000000001e-05 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454818+0000", 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.95286919199999998 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454827+0000", 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3130000000000004e-06 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454959+0000", 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013207099999999999 2024-09-17T17:31:07.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.454969+0000", 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0368e-05 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455761+0000", 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000792051 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.455800+0000", 2024-09-17T17:31:07.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8772000000000001e-05 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=123, reqid=client.4566.0:616, at_version=0'0, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 48.212011441999998, 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015260055, 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449812+0000", 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449813+0000", 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5499999999999998e-07 2024-09-17T17:31:07.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.449823+0000", 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9539999999999999e-06 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464497+0000", 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014673699 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464503+0000", 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1770000000000003e-06 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464557+0000", 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4571999999999999e-05 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.464568+0000", 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0386e-05 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465032+0000", 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046443199999999997 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.465072+0000", 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0280000000000001e-05 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=125, reqid=client.4566.0:619, at_version=68'49, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 48.195488019000003, 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010346985, 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466336+0000", 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.466642+0000", 2024-09-17T17:31:07.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030544600000000002 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475441+0000", 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087995179999999992 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475448+0000", 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4439999999999996e-06 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475494+0000", 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000003e-05 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.475500+0000", 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476652+0000", 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001152887 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476683+0000", 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0306000000000001e-05 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 68/67 ECSubWrite(tid=127, reqid=client.4566.0:621, at_version=68'51, trim_to=0'0, roll_forward_to=68'46))", 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 48.185751918999998, 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86543154899999997, 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476072+0000", 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476073+0000", 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4900000000000005e-07 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.476093+0000", 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0431000000000001e-05 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340542+0000", 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.86444875899999996 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340554+0000", 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2017e-05 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340635+0000", 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1500000000000002e-05 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.340651+0000", 2024-09-17T17:31:07.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5444999999999999e-05 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341467+0000", 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00081634899999999996 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.341503+0000", 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6298999999999999e-05 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.703 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:644 3.3s0 3:c77cc1f5:::smithi10238019-3:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.962730278999999, 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.524887887, 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 644 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699094+0000", 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699095+0000", 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.099e-06 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699100+0000", 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1549999999999998e-06 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699109+0000", 2024-09-17T17:31:07.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1670000000000005e-06 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699117+0000", 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8720000000000002e-06 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699171+0000", 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4237999999999998e-05 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699278+0000", 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107339 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.699941+0000", 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066276400000000004 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:19.702383+0000", 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024413550000000001 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223916+0000", 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52153379200000005 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.223981+0000", 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5105999999999996e-05 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.9s2 69/67 ECSubWrite(tid=129, reqid=client.4566.0:621, at_version=0'0, trim_to=0'0, roll_forward_to=68'51))", 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.319035909, 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014148114, 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.710 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342788+0000", 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342789+0000", 2024-09-17T17:31:07.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.13e-06 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.342803+0000", 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4309e-05 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356208+0000", 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013404337000000001 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356219+0000", 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1569000000000001e-05 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356293+0000", 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3894999999999996e-05 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356301+0000", 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5159999999999997e-06 2024-09-17T17:31:07.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356892+0000", 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059179499999999999 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:20.356936+0000", 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3563000000000003e-05 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:725 3.3s0 3:c5fa5218:::smithi10238019-30:head [create,setxattr n\\kYkdan\\thNN\\sTap[lWfmsQQctdU^aYUdXcPaN (278) in=40b,setxattr gdposbmqnPajgmVYW]VdX\\`gUQkaeU]UtbYUfiNg (1750) in=40b,setxattr ciVa^RQRQUnlPcOS[RhXiP[f`fVknjrV_`VY^mUX (417) in=40b,setxattr d_hhtYW (11296) in=7b,setxattr sRYR``_hO_QkVYhhONZZjoiUtaoqebe^l]rOq[_Y (1682) in=40b,setxattr _QNRabNbNqfbhqjYjpV`OpcjWYck]PSYrhXT_pR` (350) in=40b,setxattr Orajpk`Qfg^nQasaZW_ePrgt^]N__bT[NVS]PaaW (19224) in=40b,setxattr e[aOkpoq_NTqbYdOapYpT (11296) in=21b,setxattr ogiNjhNa]OiT]hhXbR_ti`jYmsgbs_rtq_jR`[`g (979) in=40b,setxattr j (11296) in=1b,setxattr N\\i[nSgaepe]]Wrqk[^`enQo]SlraVR^o_SO (11296) in=36b,setxattr cUpO`i`Te\\O\\Wtmgsdpghq]jam\\WhaNdZTYgepZZ (1049) in=40b,setxattr tU[pQ\\_U_siq^t[VT_bha\\Qjp`Y_n``SZlnPr`Nj (1329) in=40b,setxattr trgNslOknYrnhc`^OllRjkWjZc (11296) in=26b,setxattr ofS\\dVZbbra]Vf^gm[[Y^^XW`gWa_PjOdRlafjTa (66) in=40b,setxattr k^Ue`eUT^[jiTlg\\aPXs`ZXXaejRaY^X^\\oQ_clj (1262) in=40b,setxattr _goNc]^eaWViaTbdiVPjTWXpe_VrQgcY]nSajfQk (242) in=40b,setxattr sfdRTqXeR]_gb]lnNOUWXWfc\\NQcbOaTrTk`aOQq (700) in=40b,setxattr Yrh^hjspm\\YXTjTb]TRjpnsQU]sq\\c`RcfSeOchd (7436) in=40b,setxattr aahttaRSQ\\X`]orTWZRe[_OplTjd`YRoTQdeTmmj (1926) in=40b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 42.324159760999997, 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0205677150000001, 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 725 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337664+0000", 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337666+0000", 2024-09-17T17:31:07.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.513e-06 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337795+0000", 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000128961 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337808+0000", 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3713000000000001e-05 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.337819+0000", 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.027e-05 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341249+0000", 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034306190000000002 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:25.341253+0000", 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4410000000000002e-06 2024-09-17T17:31:07.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.346358+0000", 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.005105844 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.346550+0000", 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019203699999999999 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.347606+0000", 2024-09-17T17:31:07.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001055896 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.353654+0000", 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0060472520000000004 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358123+0000", 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044693620000000002 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.358232+0000", 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108807 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 75/74 ECSubWriteReply(tid=94, last_complete=75'58, committed=1, applied=1))", 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 41.306680012999998, 2024-09-17T17:31:07.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.74451308800000005, 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:31:07.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355144+0000", 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:26.355148+0000", 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.58e-06 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098760+0000", 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.74361193699999995 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098773+0000", 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.288e-05 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.098921+0000", 2024-09-17T17:31:07.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014810999999999999 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:27.099657+0000", 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00073597300000000001 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:784 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~720896] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:31:07.725 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.310351478000001, 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010356648, 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 784 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351472+0000", 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.351474+0000", 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.562e-06 2024-09-17T17:31:07.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352295+0000", 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00082097799999999999 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352310+0000", 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5599000000000001e-05 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352317+0000", 2024-09-17T17:31:07.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0900000000000001e-06 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352366+0000", 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9768999999999997e-05 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.352456+0000", 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0044999999999996e-05 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.353483+0000", 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001026788 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.357386+0000", 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039030559999999998 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361725+0000", 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043392559999999997 2024-09-17T17:31:07.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361829+0000", 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010350500000000001 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:785 3.13s0 3:c80ea1ca:::smithi10238019-27:head [append 0~622592] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.306781850999997, 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010812743, 2024-09-17T17:31:07.731 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 785 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355042+0000", 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355044+0000", 2024-09-17T17:31:07.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1919999999999999e-06 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355710+0000", 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066595000000000001 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355726+0000", 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6030999999999999e-05 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355735+0000", 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7390000000000005e-06 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355799+0000", 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4281000000000003e-05 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.355877+0000", 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7553999999999996e-05 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.356754+0000", 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087738099999999999 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.361867+0000", 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0051130380000000003 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365763+0000", 2024-09-17T17:31:07.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003895496 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365855+0000", 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2081000000000001e-05 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:788 3.13s0 3:c80ea1ca:::smithi10238019-27:head [setxattr _header (57) in=7b] snapc 19=[19,18,17,16,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:31:07.737 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.300905266000001, 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014979750999999999, 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 788 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360919+0000", 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4000000000000002e-07 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360925+0000", 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4010000000000002e-06 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360932+0000", 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1879999999999999e-06 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360937+0000", 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4330000000000003e-06 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.364485+0000", 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035482949999999999 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.364590+0000", 2024-09-17T17:31:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104424 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.365073+0000", 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048325000000000001 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.367135+0000", 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002061677 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375816+0000", 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086812480000000008 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.375898+0000", 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2294999999999997e-05 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:789 3.13s0 3:c80ea1ca:::smithi10238019-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-17T17:31:07.743 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.300878455000003, 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016800492, 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 789 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360945+0000", 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360946+0000", 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4300000000000003e-07 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360950+0000", 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.25e-06 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360958+0000", 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5360000000000004e-06 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.360963+0000", 2024-09-17T17:31:07.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5509999999999999e-06 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366330+0000", 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053669410000000001 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.366434+0000", 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103384 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.376442+0000", 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010008349 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.376609+0000", 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000166732 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:30.377746+0000", 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001137206 2024-09-17T17:31:07.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.12s5 85/84 ECSubRead(tid=64, to_read={3:4c833192:::smithi10238019-39:head=0,262144,0}, subchunks={3:4c833192:::smithi10238019-39:head=[0,1]}, attrs_to_read=))", 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.281252493, 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.043000999999999998, 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380571+0000", 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380573+0000", 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.576e-06 2024-09-17T17:31:07.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.380584+0000", 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1195e-05 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422791+0000", 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.042206670000000002 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422808+0000", 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7677e-05 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.422941+0000", 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013300900000000001 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:36.423572+0000", 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000630873 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.752 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:925 3.3s0 3:c0b80cc2:::smithi10238019-19:head [copy-from ver 153] snapc 20=[20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.345038+0000", 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.31678629, 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.041407069999999997, 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 925 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345038+0000", 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345038+0000", 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345039+0000", 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.486e-06 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345049+0000", 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0370000000000001e-05 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345074+0000", 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4453000000000001e-05 2024-09-17T17:31:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345083+0000", 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5859999999999994e-06 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345145+0000", 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1983000000000006e-05 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345160+0000", 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4321000000000001e-05 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.352926+0000", 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077658359999999999 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.353042+0000", 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011646699999999999 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.380319+0000", 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027276584 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.384772+0000", 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004453785 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.386372+0000", 2024-09-17T17:31:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016000960000000001 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.386445+0000", 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2102999999999997e-05 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:926 3.3s0 3:c0b80cc2:::smithi10238019-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.345110+0000", 2024-09-17T17:31:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.316713602, 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.037529039, 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 926 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345110+0000", 2024-09-17T17:31:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345110+0000", 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345111+0000", 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1299999999999999e-07 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345115+0000", 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7689999999999998e-06 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345123+0000", 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345130+0000", 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8759999999999999e-06 2024-09-17T17:31:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345224+0000", 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4067000000000006e-05 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.345232+0000", 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3939999999999998e-06 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.353253+0000", 2024-09-17T17:31:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080215349999999998 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.353329+0000", 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5767999999999997e-05 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.382446+0000", 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.029117214999999998 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.382536+0000", 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9852000000000006e-05 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.382639+0000", 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010304799999999999 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:934 3.3s0 3:c5fa5218:::smithi10238019-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:48.360510+0000", 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.301313359000002, 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012112242, 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 934 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360510+0000", 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360510+0000", 2024-09-17T17:31:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360511+0000", 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.37e-07 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360514+0000", 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3790000000000002e-06 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360522+0000", 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9160000000000001e-06 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360528+0000", 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7100000000000004e-06 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360619+0000", 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1333000000000001e-05 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.360623+0000", 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8850000000000001e-06 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.365886+0000", 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052631880000000002 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.365996+0000", 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109456 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.370760+0000", 2024-09-17T17:31:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047638439999999997 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.370864+0000", 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103968 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:48.372623+0000", 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0017589260000000001 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:07.773 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:07.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-17T17:31:07.881 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:07.881 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-17T17:31:07.881 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:07.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-17T17:31:07.996 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-17T17:31:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-17T17:31:07.996 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-17T17:31:07.996 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:08.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-17T17:31:08.112 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-17T17:31:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-17T17:31:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:308 3.2s0 3:4c833192:::smithi10238019-39:head [rollback 5] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.139459474000006, 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076693619999999999, 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 308 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970364+0000", 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.68e-07 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970369+0000", 2024-09-17T17:31:08.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7160000000000002e-06 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970378+0000", 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5739999999999996e-06 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970383+0000", 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0010000000000001e-06 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971568+0000", 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011857020000000001 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.971627+0000", 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8826999999999998e-05 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972103+0000", 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047574100000000001 2024-09-17T17:31:08.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973681+0000", 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001578404 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.977986+0000", 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043046630000000002 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978033+0000", 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7166000000000002e-05 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:309 3.2s0 3:4c833192:::smithi10238019-39:head [append 0~0] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.139428432000003, 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077796990000000002, 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 309 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:31:08.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970395+0000", 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2e-07 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970399+0000", 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0779999999999997e-06 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970408+0000", 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7979999999999995e-06 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.970415+0000", 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9310000000000002e-06 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972569+0000", 2024-09-17T17:31:08.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021534969999999999 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.972677+0000", 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108145 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973250+0000", 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057328100000000003 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.973923+0000", 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00067244899999999999 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978120+0000", 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041977949999999998 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:54.978175+0000", 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4205e-05 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.124 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:327 3.2s0 3:497089da:::smithi10238019-9:head [rollback 5] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.410974010999993, 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0072194929999999996, 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.125 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 327 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698849+0000", 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698850+0000", 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698852+0000", 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3939999999999999e-06 2024-09-17T17:31:08.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698858+0000", 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1480000000000003e-06 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698863+0000", 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3470000000000001e-06 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698988+0000", 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012491999999999999 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698997+0000", 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6819999999999998e-06 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702506+0000", 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035088879999999999 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.702618+0000", 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111428 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703259+0000", 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064172000000000003 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704549+0000", 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001289599 2024-09-17T17:31:08.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706021+0000", 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001471857 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706069+0000", 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8072000000000003e-05 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:328 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.410934468999997, 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0073597310000000004, 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 328 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.132 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698889+0000", 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7e-07 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698894+0000", 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.057e-06 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698902+0000", 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3760000000000002e-06 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.698907+0000", 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0329999999999999e-06 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699035+0000", 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012849700000000001 2024-09-17T17:31:08.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.699042+0000", 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.55e-06 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703631+0000", 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004589536 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.703735+0000", 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103891 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704152+0000", 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041683699999999998 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.704821+0000", 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066855199999999999 2024-09-17T17:31:08.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706206+0000", 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001385788 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:29:56.706249+0000", 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2243999999999997e-05 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:421 3.2s0 3:4c833192:::smithi10238019-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.727182244999995, 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090389679999999997, 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 421 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.139 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382641+0000", 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382642+0000", 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000005e-07 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382644+0000", 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382652+0000", 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5059999999999998e-06 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.382656+0000", 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8029999999999996e-06 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385875+0000", 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0032189900000000001 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.385954+0000", 2024-09-17T17:31:08.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9087999999999999e-05 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388823+0000", 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028687090000000001 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.388909+0000", 2024-09-17T17:31:08.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5457000000000005e-05 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:03.391680+0000", 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027713939999999999 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 53/35 ECSubWrite(tid=133, reqid=osd.13.0:133, at_version=53'64, trim_to=0'0, roll_forward_to=52'62))", 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.747779059999999, 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078716470000000007, 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362044+0000", 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4499999999999998e-07 2024-09-17T17:31:08.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362068+0000", 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3370000000000002e-05 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362467+0000", 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039876899999999999 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.362476+0000", 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8740000000000001e-06 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364044+0000", 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001568261 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.364055+0000", 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.077e-05 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369861+0000", 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0058066660000000003 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:04.369916+0000", 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4691999999999997e-05 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.149 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s5 56/35 ECSubWrite(tid=87, reqid=osd.12.0:87, at_version=56'34, trim_to=0'0, roll_forward_to=53'30))", 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 60.770405920999998, 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074908880000000002, 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339417+0000", 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339418+0000", 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2399999999999999e-07 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339449+0000", 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.133e-05 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339554+0000", 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104682 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339563+0000", 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9129999999999992e-06 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339937+0000", 2024-09-17T17:31:08.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037396800000000002 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.339947+0000", 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0451999999999999e-05 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346872+0000", 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0069250020000000004 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:07.346908+0000", 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6117e-05 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=154, reqid=osd.13.0:154, at_version=57'84, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.719908449000002, 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035993311, 2024-09-17T17:31:08.154 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389915+0000", 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389916+0000", 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0299999999999997e-07 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.389936+0000", 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0061e-05 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425016+0000", 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.035080622999999998 2024-09-17T17:31:08.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425025+0000", 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3110000000000006e-06 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425088+0000", 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3442999999999995e-05 2024-09-17T17:31:08.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425103+0000", 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4625e-05 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425875+0000", 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000772585 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.425908+0000", 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2759999999999998e-05 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 57/35 ECSubWrite(tid=155, reqid=osd.13.0:155, at_version=57'86, trim_to=0'0, roll_forward_to=57'82))", 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.719007269999999, 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26589276499999998, 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390816+0000", 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390817+0000", 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.390839+0000", 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2725999999999999e-05 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655507+0000", 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26466748600000001 2024-09-17T17:31:08.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655515+0000", 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5530000000000004e-06 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655625+0000", 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109997 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655635+0000", 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0000000000000001e-05 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656673+0000", 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001038464 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.656709+0000", 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5830000000000001e-05 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.163 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=156, reqid=client.4566.0:492, at_version=57'10, trim_to=0'0, roll_forward_to=43'8))", 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.684846794999999, 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23215693400000001, 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424977+0000", 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4900000000000004e-07 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.424998+0000", 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0800000000000001e-05 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655688+0000", 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.23068955499999999 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655696+0000", 2024-09-17T17:31:08.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0839999999999992e-06 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655818+0000", 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012232399999999999 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.655830+0000", 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1355000000000001e-05 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657105+0000", 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001275403 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657133+0000", 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8464000000000001e-05 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 57/32 ECSubWrite(tid=158, reqid=client.4566.0:492, at_version=0'0, trim_to=0'0, roll_forward_to=57'10))", 2024-09-17T17:31:08.168 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.452001731999999, 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26690073399999997, 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657822+0000", 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000001e-07 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.657834+0000", 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1450999999999999e-05 2024-09-17T17:31:08.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923785+0000", 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.26595101999999998 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923800+0000", 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.508e-05 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923956+0000", 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155876 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.923975+0000", 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9443e-05 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924688+0000", 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00071237099999999995 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:08.924722+0000", 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4629000000000003e-05 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.6s1 59/35 ECSubWrite(tid=164, reqid=client.4566.0:513, at_version=59'90, trim_to=0'0, roll_forward_to=59'88))", 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.734281054, 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080861479999999996, 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375542+0000", 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375543+0000", 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4499999999999995e-07 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375918+0000", 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037515399999999999 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375961+0000", 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2206000000000002e-05 2024-09-17T17:31:08.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.375969+0000", 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8349999999999993e-06 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376050+0000", 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0269e-05 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.376060+0000", 2024-09-17T17:31:08.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0509999999999999e-05 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383584+0000", 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075235939999999998 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:10.383628+0000", 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4635000000000002e-05 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:562 3.2s0 3:497089da:::smithi10238019-9:head [rollback f] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 53.693689722999999, 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076874359999999997, 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 562 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416134+0000", 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1299999999999999e-07 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416138+0000", 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4920000000000002e-06 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416146+0000", 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8800000000000008e-06 2024-09-17T17:31:08.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416150+0000", 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6840000000000004e-06 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416271+0000", 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012026799999999999 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416278+0000", 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5889999999999997e-06 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419622+0000", 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0033434419999999999 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.419740+0000", 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011798600000000001 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420292+0000", 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00055223900000000005 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421629+0000", 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001337429 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423776+0000", 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021470370000000001 2024-09-17T17:31:08.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423821+0000", 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4676999999999999e-05 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:563 3.2s0 3:497089da:::smithi10238019-9:head [append 0~0] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 53.693665871, 2024-09-17T17:31:08.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079319400000000002, 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 563 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416157+0000", 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416158+0000", 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416161+0000", 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6649999999999999e-06 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416171+0000", 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0176e-05 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416181+0000", 2024-09-17T17:31:08.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0327e-05 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416310+0000", 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129247 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.416317+0000", 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0720000000000002e-06 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420645+0000", 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043279629999999998 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.420731+0000", 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5937000000000001e-05 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.421175+0000", 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044333000000000003 2024-09-17T17:31:08.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.423834+0000", 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002659121 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424046+0000", 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021196999999999999 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:14.424089+0000", 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3702999999999998e-05 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:907 3.12s0 3:497089da:::smithi10238019-9:head [copy-from ver 0] snapc 1f=[1f,1e,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.372716+0000", 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.737107731000002, 2024-09-17T17:31:08.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079506950000000007, 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 907 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372716+0000", 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372716+0000", 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372718+0000", 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372740+0000", 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2580000000000001e-05 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372766+0000", 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5984000000000001e-05 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372784+0000", 2024-09-17T17:31:08.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7949e-05 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372887+0000", 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102662 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373071+0000", 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018410400000000001 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380666+0000", 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0075953660000000001 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4566.0:908 3.12s0 3:497089da:::smithi10238019-9:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:42.372809+0000", 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.737014394999999, 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.008374968, 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4566", 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 908 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372809+0000", 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372809+0000", 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372810+0000", 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.372813+0000", 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8199999999999998e-06 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373013+0000", 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019992100000000001 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373022+0000", 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8089999999999999e-06 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373804+0000", 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00078201900000000001 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.373866+0000", 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1671e-05 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.380885+0000", 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0070188480000000003 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.381046+0000", 2024-09-17T17:31:08.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016086799999999999 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:42.381184+0000", 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013826600000000001 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.0s5 92/75 ECSubWrite(tid=212, reqid=osd.12.0:212, at_version=92'80, trim_to=0'0, roll_forward_to=91'76))", 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.755673421000001, 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0089937200000000002, 2024-09-17T17:31:08.201 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354150+0000", 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7899999999999999e-07 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354174+0000", 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3799999999999999e-05 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354419+0000", 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024507199999999999 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354438+0000", 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8819000000000001e-05 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354944+0000", 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050620199999999998 2024-09-17T17:31:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.354954+0000", 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.009e-05 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363090+0000", 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081354419999999997 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.363144+0000", 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3915999999999997e-05 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s2 92/74 ECSubWrite(tid=316, reqid=osd.13.0:316, at_version=92'34, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.356287+0000", 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.753536344, 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012103668, 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356287+0000", 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356287+0000", 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356288+0000", 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8899999999999998e-07 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.356316+0000", 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7753e-05 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358713+0000", 2024-09-17T17:31:08.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023969759999999999 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358722+0000", 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9139999999999997e-06 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358826+0000", 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010438 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358838+0000", 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1817000000000001e-05 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.368219+0000", 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093809289999999997 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.368391+0000", 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017191 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.1s2 92/74 ECSubWrite(tid=317, reqid=osd.13.0:317, at_version=92'36, trim_to=0'0, roll_forward_to=91'32))", 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.750991806999998, 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.096070654000000005, 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358832+0000", 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1100000000000002e-07 2024-09-17T17:31:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.358851+0000", 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8972999999999999e-05 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453479+0000", 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.094628491999999995 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453495+0000", 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498999999999998e-05 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453664+0000", 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016951999999999999 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.453697+0000", 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2301999999999997e-05 2024-09-17T17:31:08.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.454861+0000", 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001164099 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:43.454902+0000", 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1458000000000002e-05 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.bs6 96/95 ECSubWrite(tid=332, reqid=osd.13.0:332, at_version=96'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-17T17:30:47.362945+0000", 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.746878216999999, 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081654910000000004, 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362945+0000", 2024-09-17T17:31:08.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362945+0000", 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362946+0000", 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362964+0000", 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8206000000000002e-05 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362992+0000", 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7773e-05 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.362999+0000", 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7780000000000007e-06 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.363314+0000", 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00031452100000000001 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.363324+0000", 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8050000000000001e-06 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.371074+0000", 2024-09-17T17:31:08.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.007750458 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-17T17:30:47.371111+0000", 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6439999999999997e-05 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-17T17:31:08.220 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-17T17:31:08.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /var/log/ceph/exp.3.bs4.8 2024-09-17T17:31:08.223 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T17:31:08.335 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:08.336 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:08.336 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:31:08.336 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:08.344 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T17:31:08.448 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:08.448 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:08.448 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:31:08.448 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:31:08.449 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:08.457 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T17:31:08.562 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:169 3.as0 3:5ee91470:::smithi10238019-6:head [read 83682~14622,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 79.569309836000002, 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67448544799999999, 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 169 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.564 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990277+0000", 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990278+0000", 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6400000000000002e-07 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.565 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990283+0000", 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3900000000000001e-06 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990291+0000", 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9379999999999992e-06 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.566 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990297+0000", 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6099999999999997e-06 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992308+0000", 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0020111650000000001 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.567 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992380+0000", 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1459999999999997e-05 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.664763+0000", 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67238332099999998 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.568 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.569 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.569 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.569 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 37/32 ECSubWriteReply(tid=24, last_complete=37'12, committed=1, applied=1))", 2024-09-17T17:31:08.569 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:31:08.569 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 79.559277993999999, 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069166769000000003, 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.570 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000309+0000", 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000310+0000", 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1299999999999998e-07 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.571 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000314+0000", 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.461e-06 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069373+0000", 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.069059000999999995 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.572 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069380+0000", 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2570000000000001e-06 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069437+0000", 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7519999999999998e-05 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.573 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069476+0000", 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8617e-05 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/35 ECSubWrite(tid=18, reqid=client.4566.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=37'13))", 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 79.490243677999999, 2024-09-17T17:31:08.574 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52672497699999998, 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.575 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069344+0000", 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069345+0000", 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.048e-06 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.576 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069355+0000", 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0052e-05 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595186+0000", 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.52583139400000001 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595201+0000", 2024-09-17T17:31:08.577 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4790000000000001e-05 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595348+0000", 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014743699999999999 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.595363+0000", 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4589999999999999e-05 2024-09-17T17:31:08.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596033+0000", 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067039499999999995 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.596069+0000", 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5271000000000002e-05 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.579 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:398 3.as0 3:5280ab18:::smithi10238019-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.216279456999999, 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013718741, 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.580 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 398 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.581 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343308+0000", 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9700000000000003e-07 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.582 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343310+0000", 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.525e-06 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343314+0000", 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.8770000000000003e-06 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.583 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.343316+0000", 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8080000000000001e-06 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347523+0000", 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042062560000000002 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.347583+0000", 2024-09-17T17:31:08.584 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0201999999999997e-05 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356234+0000", 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086509659999999995 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.356431+0000", 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00019758699999999999 2024-09-17T17:31:08.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.357027+0000", 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00059522299999999998 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 51/32 ECSubWriteReply(tid=64, last_complete=51'43, committed=1, applied=1))", 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:31:08.586 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.202812949000005, 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.78010271600000003, 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356774+0000", 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356775+0000", 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5000000000000001e-07 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.588 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356781+0000", 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.716e-06 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136139+0000", 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.77935822499999996 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.589 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136153+0000", 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.359e-05 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136325+0000", 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00017257700000000001 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.136877+0000", 2024-09-17T17:31:08.590 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00055165800000000001 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:641 3.as0 3:5280ab18:::smithi10238019-18:head [delete] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.035889339000001, 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.16562932699999999, 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.591 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 641 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:31:08.592 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523698+0000", 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523700+0000", 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1150000000000001e-06 2024-09-17T17:31:08.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523708+0000", 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5599999999999996e-06 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.594 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523725+0000", 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7725999999999999e-05 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523733+0000", 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5299999999999999e-06 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.595 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6327000000000004e-05 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523914+0000", 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114837 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524640+0000", 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072579000000000003 2024-09-17T17:31:08.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526382+0000", 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0017423060000000001 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689252+0000", 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.162869823 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.597 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689327+0000", 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5313000000000002e-05 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:661 3.as0 3:5a070e56:::smithi10238019-47:head [copy-from ver 88] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 48.212411295999999, 2024-09-17T17:31:08.598 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.026523057999999999, 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 661 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.599 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347176+0000", 2024-09-17T17:31:08.600 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347177+0000", 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0999999999999999e-07 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347181+0000", 2024-09-17T17:31:08.601 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5050000000000001e-06 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347187+0000", 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1990000000000003e-06 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347192+0000", 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2969999999999997e-06 2024-09-17T17:31:08.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347359+0000", 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016719099999999999 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347361+0000", 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7039999999999999e-06 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.603 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351847+0000", 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.004485274 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.351930+0000", 2024-09-17T17:31:08.604 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.2813999999999997e-05 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.367807+0000", 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015877189999999999 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.370285+0000", 2024-09-17T17:31:08.605 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0024778510000000001 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.373602+0000", 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0033175460000000002 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.373699+0000", 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6977000000000003e-05 2024-09-17T17:31:08.606 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:662 3.as0 3:5a070e56:::smithi10238019-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 48.212339016999998, 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.022338108999999998, 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.607 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 662 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:31:08.608 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347248+0000", 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347249+0000", 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2099999999999997e-07 2024-09-17T17:31:08.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347251+0000", 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6589999999999999e-06 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347258+0000", 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4330000000000001e-06 2024-09-17T17:31:08.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347263+0000", 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3979999999999998e-06 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347385+0000", 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000121845 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.611 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347390+0000", 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.147e-06 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352716+0000", 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053252270000000001 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.612 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352767+0000", 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1344999999999997e-05 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369400+0000", 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016633254 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.613 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369488+0000", 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.7441999999999994e-05 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.369586+0000", 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8838000000000005e-05 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.614 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:663 3.as0 3:5ee91470:::smithi10238019-6:head [delete] snapc 14=[14,13,11,10,f] ondisk+write+known_if_redirected+full_force+supports_pool_eio e69)", 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 48.212316637000001, 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016232469999999999, 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 663 2024-09-17T17:31:08.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.616 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347271+0000", 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1300000000000001e-07 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347273+0000", 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2800000000000002e-06 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.617 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347279+0000", 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2170000000000002e-06 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347283+0000", 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4719999999999997e-06 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.618 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347416+0000", 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013269700000000001 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.347421+0000", 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1019999999999996e-06 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352795+0000", 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053736970000000002 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.352873+0000", 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8318000000000005e-05 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.620 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.353539+0000", 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00066570400000000004 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.356943+0000", 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034039679999999998 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.621 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.363452+0000", 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.006509384 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.363503+0000", 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1218000000000003e-05 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.622 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.as0 73/32 ECSubWriteReply(tid=139, last_complete=73'108, committed=1, applied=1))", 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.817369505999999, 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.257005174, 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.624 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742218+0000", 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2399999999999998e-07 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.742222+0000", 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4359999999999998e-06 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.625 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998674+0000", 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.25645215799999999 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998683+0000", 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1600000000000004e-06 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.626 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.998749+0000", 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5689000000000001e-05 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:24.999223+0000", 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047410699999999998 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.627 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:736 3.as0 3:5ceb0529:::smithi10238019-12:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.175602724999997, 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.95058956999999999, 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.628 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 736 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.629 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383985+0000", 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.44e-07 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.630 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383988+0000", 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.926e-06 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.383994+0000", 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2729999999999999e-06 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384000+0000", 2024-09-17T17:31:08.631 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.237e-06 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385207+0000", 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001207079 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385300+0000", 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3450999999999997e-05 2024-09-17T17:31:08.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385740+0000", 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000440312 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386613+0000", 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00087278900000000001 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.633 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334485+0000", 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94787173599999996 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334574+0000", 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9222999999999996e-05 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:737 3.as0 3:5ceb0529:::smithi10238019-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.175579826000003, 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.95178945199999998, 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.635 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 737 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.636 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384007+0000", 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384008+0000", 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5299999999999999e-07 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384011+0000", 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.852e-06 2024-09-17T17:31:08.637 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384018+0000", 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3860000000000001e-06 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.384023+0000", 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7790000000000003e-06 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.638 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386679+0000", 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0026562550000000002 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.386739+0000", 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0207999999999997e-05 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.639 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334966+0000", 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94822698000000005 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.335064+0000", 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7929000000000004e-05 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.335797+0000", 2024-09-17T17:31:08.640 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00073260999999999997 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.10.22:3 3.16s0 3:6d14b604:::smithi10238019-45:head [assert-version v153,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.359098+0000", 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.200488928999999, 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015040585, 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.641 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359098+0000", 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359098+0000", 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359100+0000", 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.57e-06 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359106+0000", 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6219999999999997e-06 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.643 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359119+0000", 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3735e-05 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.644 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359130+0000", 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.022e-05 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359299+0000", 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016968300000000001 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for peered", 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.359303+0000", 2024-09-17T17:31:08.645 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0369999999999996e-06 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.365427+0000", 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0061239780000000004 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.365942+0000", 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00051429299999999998 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.646 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.374139+0000", 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0081974470000000001 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 96/95 ECSubWriteReply(tid=175, last_complete=96'137, committed=1, applied=1))", 2024-09-17T17:31:08.647 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.181773540000002, 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089945729000000002, 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377814+0000", 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4600000000000004e-07 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.649 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.377819+0000", 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3170000000000003e-06 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467660+0000", 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089841455000000001 2024-09-17T17:31:08.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467686+0000", 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5420999999999999e-05 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467743+0000", 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7120000000000002e-05 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.651 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.467759+0000", 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.677e-05 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 96/95 ECSubWriteReply(tid=175, last_complete=96'137, committed=1, applied=1))", 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.378380+0000", 2024-09-17T17:31:08.652 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.181207415999999, 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.71966408999999998, 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.378380+0000", 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.653 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.378380+0000", 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.378381+0000", 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0399999999999995e-07 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.654 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.378385+0000", 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9400000000000004e-06 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.097927+0000", 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.71954220999999996 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.655 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.097939+0000", 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2727e-05 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.098008+0000", 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8907999999999998e-05 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.656 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.098044+0000", 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5601e-05 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:935 3.6s0 3:675a7e9e:::smithi10238019-10:head [append 0~278528] snapc 21=[21,20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.361975+0000", 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 20.197612553999999, 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014530448, 2024-09-17T17:31:08.657 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 935 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.658 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.361975+0000", 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.361975+0000", 2024-09-17T17:31:08.659 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.361977+0000", 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5890000000000001e-06 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362137+0000", 2024-09-17T17:31:08.660 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015933899999999999 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362157+0000", 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0641000000000001e-05 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362167+0000", 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5999999999999996e-06 2024-09-17T17:31:08.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362236+0000", 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8745000000000007e-05 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362243+0000", 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.0859999999999996e-06 2024-09-17T17:31:08.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364757+0000", 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0025138959999999998 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364891+0000", 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013427 2024-09-17T17:31:08.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.365705+0000", 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00081406200000000005 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.369653+0000", 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0039480749999999997 2024-09-17T17:31:08.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.376420+0000", 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0067669150000000001 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.376505+0000", 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5229999999999995e-05 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.665 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:936 3.6s0 3:675a7e9e:::smithi10238019-10:head [setxattr _header (57) in=7b] snapc 21=[21,20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 20.197397697, 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014521408, 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.666 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 936 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.667 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8800000000000002e-07 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.668 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362194+0000", 2024-09-17T17:31:08.669 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7409999999999998e-06 2024-09-17T17:31:08.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.669 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.669 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362204+0000", 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.023e-05 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362208+0000", 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.9330000000000003e-06 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362313+0000", 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010539699999999999 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362319+0000", 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0100000000000003e-06 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.367209+0000", 2024-09-17T17:31:08.671 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048908320000000003 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.367317+0000", 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107775 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.367773+0000", 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00045616100000000001 2024-09-17T17:31:08.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.369656+0000", 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018825140000000001 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.376629+0000", 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0069735910000000003 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.673 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.376711+0000", 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1636000000000002e-05 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:937 3.6s0 3:675a7e9e:::smithi10238019-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362240+0000", 2024-09-17T17:31:08.674 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 20.197346914000001, 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016183497000000002, 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 937 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362240+0000", 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362240+0000", 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362241+0000", 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.47e-07 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362245+0000", 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2320000000000003e-06 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362255+0000", 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4970000000000007e-06 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362262+0000", 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3969999999999997e-06 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362338+0000", 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5889999999999993e-05 2024-09-17T17:31:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362342+0000", 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5039999999999998e-06 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.368413+0000", 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0060712129999999998 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.368574+0000", 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016084399999999999 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.377197+0000", 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086234899999999993 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.377325+0000", 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012811800000000001 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.378424+0000", 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0010984650000000001 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:971 3.6s0 3:624f05ea:::smithi10238019-26:head [copy-from ver 32] snapc 22=[22,21,20,18,17,10] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:51.753423+0000", 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 16.806164237000001, 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.033497621999999998, 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 971 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753423+0000", 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753423+0000", 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753425+0000", 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.657e-06 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753432+0000", 2024-09-17T17:31:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2200000000000003e-06 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753462+0000", 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0107000000000002e-05 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753475+0000", 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2594e-05 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753538+0000", 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2940000000000004e-05 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753649+0000", 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111325 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.773333+0000", 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.019683948 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.777082+0000", 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0037488140000000001 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.786827+0000", 2024-09-17T17:31:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097452159999999993 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.786921+0000", 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3801000000000005e-05 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:972 3.6s0 3:624f05ea:::smithi10238019-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:51.753492+0000", 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 16.806094994999999, 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021136305000000001, 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 972 2024-09-17T17:31:08.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753492+0000", 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753492+0000", 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:08.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753493+0000", 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5499999999999998e-07 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753498+0000", 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.22e-06 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.691 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753506+0000", 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8390000000000007e-06 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753510+0000", 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.109e-06 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.692 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753850+0000", 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00033958000000000003 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753907+0000", 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7491000000000002e-05 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.774397+0000", 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.020489673999999999 2024-09-17T17:31:08.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.774506+0000", 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010882000000000001 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:08.694 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.774629+0000", 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000122917 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:08.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T17:31:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:08.800 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:08.801 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:31:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:08.808 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T17:31:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:08.911 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:08.911 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:31:08.911 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:31:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:08.919 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T17:31:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.023 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:31:09.023 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:312 3.ds0 3:badfad2a:::smithi10238019-37:5 [read 0~1736704,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 74.050368590000005, 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50817852500000005, 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.024 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 312 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970442+0000", 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1600000000000002e-07 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970445+0000", 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7999999999999999e-06 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970451+0000", 2024-09-17T17:31:09.026 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0970000000000002e-06 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.970456+0000", 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6009999999999997e-06 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.027 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976257+0000", 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005800889 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.976348+0000", 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1614999999999996e-05 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478621+0000", 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.50227210700000002 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.028 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.ds0 43/32 ECSubReadReply(tid=23, attrs_read=0))", 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 74.043757897999996, 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.501606106, 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.029 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977053+0000", 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.977054+0000", 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.16e-07 2024-09-17T17:31:09.030 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.978014+0000", 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096069199999999997 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477737+0000", 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.49972330100000001 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.031 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477745+0000", 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2130000000000002e-06 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.477802+0000", 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7555e-05 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.478659+0000", 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00085652899999999997 2024-09-17T17:31:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=31, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 68.662939765999994, 2024-09-17T17:31:09.033 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53920807500000001, 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:31:09.034 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357871+0000", 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.357876+0000", 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.092e-06 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.035 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896536+0000", 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53865993499999998 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896547+0000", 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1608e-05 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.896653+0000", 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000105807 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.036 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897079+0000", 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042597099999999998 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 49/32 ECSubWriteReply(tid=32, last_complete=49'23, committed=1, applied=1))", 2024-09-17T17:31:09.037 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 68.123308597000005, 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222760399, 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.038 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897502+0000", 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897503+0000", 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.68e-07 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:00.897507+0000", 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7709999999999997e-06 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.039 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120082+0000", 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.222574243 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120095+0000", 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.291e-05 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.040 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120199+0000", 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010402499999999999 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:01.120262+0000", 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3781999999999993e-05 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.041 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:412 3.ds0 3:badfad2a:::smithi10238019-37:head [append 0~655360] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.664093786999999, 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097729180000000002, 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.042 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 412 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356717+0000", 2024-09-17T17:31:09.043 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.356719+0000", 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.255e-06 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357255+0000", 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00053548099999999998 2024-09-17T17:31:09.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357270+0000", 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5045000000000001e-05 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357278+0000", 2024-09-17T17:31:09.045 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8429999999999999e-06 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357346+0000", 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7186000000000004e-05 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.046 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.357460+0000", 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000114388 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.358484+0000", 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023641 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366119+0000", 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076354270000000002 2024-09-17T17:31:09.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366416+0000", 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00029701100000000002 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366490+0000", 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3640999999999995e-05 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:415 3.ds0 3:badfad2a:::smithi10238019-37:head [setxattr _header (57) in=7b] snapc b=[b,a,9,8,6] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.658900126000006, 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089535759999999995, 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 415 2024-09-17T17:31:09.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361910+0000", 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.050 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361911+0000", 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7e-07 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361913+0000", 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2350000000000002e-06 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.051 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361919+0000", 2024-09-17T17:31:09.052 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4750000000000001e-06 2024-09-17T17:31:09.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.052 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.052 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.052 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361922+0000", 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3110000000000001e-06 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366710+0000", 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047879840000000003 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.366770+0000", 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9886000000000002e-05 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.053 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367045+0000", 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027536599999999998 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370540+0000", 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034951610000000001 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370583+0000", 2024-09-17T17:31:09.054 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2682999999999998e-05 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370864+0000", 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00028100500000000002 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:416 3.ds0 3:badfad2a:::smithi10238019-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:31:09.055 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.658883099999997, 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0096567449999999996, 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 416 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.056 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361927+0000", 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361928+0000", 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5400000000000002e-07 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.057 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361929+0000", 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.471e-06 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361934+0000", 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.1100000000000002e-06 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.361937+0000", 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1499999999999999e-06 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367591+0000", 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.005653534 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.367664+0000", 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2779000000000002e-05 2024-09-17T17:31:09.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.060 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.060 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.060 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370895+0000", 2024-09-17T17:31:09.060 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0032314409999999998 2024-09-17T17:31:09.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.370943+0000", 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7886999999999997e-05 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.371584+0000", 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000641119 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.061 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 56/32 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=56'57, trim_to=0'0, roll_forward_to=53'55))", 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 61.677190928000002, 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089122520000000007, 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.062 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.063 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343620+0000", 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.92e-07 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343638+0000", 2024-09-17T17:31:09.064 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8074e-05 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343672+0000", 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343680+0000", 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8979999999999996e-06 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.065 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343804+0000", 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012350499999999999 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.343814+0000", 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0353000000000001e-05 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352491+0000", 2024-09-17T17:31:09.066 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086771880000000006 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.352532+0000", 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0837999999999998e-05 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 74/32 ECSubWrite(tid=148, reqid=client.4566.0:736, at_version=74'114, trim_to=0'0, roll_forward_to=74'111))", 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:31:09.067 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.635095055999997, 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94858601300000001, 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.068 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385716+0000", 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5599999999999995e-07 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:25.385733+0000", 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7152000000000001e-05 2024-09-17T17:31:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.070 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.070 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333728+0000", 2024-09-17T17:31:09.070 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.94799460099999999 2024-09-17T17:31:09.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333734+0000", 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6109999999999999e-06 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.071 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333806+0000", 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1377000000000005e-05 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.333813+0000", 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2969999999999998e-06 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334272+0000", 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000458956 2024-09-17T17:31:09.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.334302+0000", 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9462999999999999e-05 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.16s5 75/74 ECSubWrite(tid=237, reqid=osd.13.0:237, at_version=75'130, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:31:09.073 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 42.656963158000003, 2024-09-17T17:31:09.074 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011160745999999999, 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.076 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363847+0000", 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363848+0000", 2024-09-17T17:31:09.077 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7400000000000003e-07 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363866+0000", 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8411999999999999e-05 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363893+0000", 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6346000000000002e-05 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.078 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363899+0000", 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5220000000000004e-06 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363943+0000", 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3933000000000001e-05 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.079 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.363948+0000", 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0810000000000001e-06 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.374983+0000", 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011034284 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.375008+0000", 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.5593999999999999e-05 2024-09-17T17:31:09.080 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 78/32 ECSubWriteReply(tid=78, last_complete=78'67, committed=1, applied=1))", 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.376561035999998, 2024-09-17T17:31:09.081 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.370820394, 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:31:09.082 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644250+0000", 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4000000000000001e-07 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.644254+0000", 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.924e-06 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.083 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014178+0000", 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.369923422 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014192+0000", 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4722999999999999e-05 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.014343+0000", 2024-09-17T17:31:09.084 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015040400000000001 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.015070+0000", 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00072728099999999998 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 78/32 ECSubWriteReply(tid=79, last_complete=78'67, committed=1, applied=1))", 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.004715611999998, 2024-09-17T17:31:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.47947288999999998, 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016095+0000", 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016096+0000", 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7000000000000005e-07 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.087 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.016099+0000", 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2660000000000002e-06 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495222+0000", 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.47912339300000001 2024-09-17T17:31:09.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495235+0000", 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3134e-05 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495445+0000", 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000209302 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.495568+0000", 2024-09-17T17:31:09.089 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000123225 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 85/84 [PushOp(3:497089da:::smithi10238019-9:13, version: 71'47, data_included: [0~36864], data_size: 36864, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:13@71'47, size: 147456, copy_subset: [], clone_subset: {}, snapset: 1b={13: [13, 10], 1b: [1b, 1a, 18, 17, 16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 147456, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.580719170999998, 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.43750746400000001, 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.090 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440091+0000", 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.091 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440092+0000", 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8999999999999996e-07 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.440171+0000", 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9203999999999996e-05 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877162+0000", 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.436990934 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.092 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877172+0000", 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0164e-05 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877272+0000", 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9265000000000006e-05 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.877599+0000", 2024-09-17T17:31:09.093 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00032730699999999999 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 85/84 [PushOp(3:497089da:::smithi10238019-9:1b, version: 82'53, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:1b@82'53, size: 819200, copy_subset: [], clone_subset: {}, snapset: 1b={13: [13, 10], 1b: [1b, 1a, 18, 17, 16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 819200, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 32.131093221999997, 2024-09-17T17:31:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.46291758199999999, 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889717+0000", 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889718+0000", 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8200000000000008e-07 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.889996+0000", 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00027726200000000002 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.096 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351714+0000", 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.46171880599999998 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351746+0000", 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1831000000000001e-05 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351912+0000", 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00016581899999999999 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.097 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.352635+0000", 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000722882 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDPGPush(3.12s1 86/84 [PushOp(3:497089da:::smithi10238019-9:head, version: 82'58, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:497089da:::smithi10238019-9:head@82'58, size: 3112960, copy_subset: [], clone_subset: {}, snapset: 0={}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 31.668989691, 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.129055472, 2024-09-17T17:31:09.098 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.099 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.351821+0000", 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8800000000000002e-07 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.352494+0000", 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00067254700000000001 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480390+0000", 2024-09-17T17:31:09.100 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.12789568900000001 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480399+0000", 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.7610000000000002e-06 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480484+0000", 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4571000000000001e-05 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.101 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:37.480876+0000", 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039231599999999998 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:886 3.ds0 3:bc68f0c8:::smithi10238019-15:head [append 0~0] snapc 1f=[1f,1e,1c,1b,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 27.680477529000001, 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089288609999999997, 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.102 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 886 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:31:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340333+0000", 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1699999999999999e-07 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340336+0000", 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3120000000000001e-06 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.104 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340342+0000", 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5590000000000002e-06 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340347+0000", 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0719999999999999e-06 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340471+0000", 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012381499999999999 2024-09-17T17:31:09.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.340475+0000", 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7270000000000001e-06 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345176+0000", 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0047006569999999996 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345272+0000", 2024-09-17T17:31:09.106 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6216000000000002e-05 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.345702+0000", 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00043059000000000001 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.346307+0000", 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00060444100000000005 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.107 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.349198+0000", 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0028914650000000002 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:41.349262+0000", 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3590000000000006e-05 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.108 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as1 91/32 ECSubWrite(tid=166, reqid=osd.3.0:166, at_version=91'134, trim_to=0'0, roll_forward_to=85'132))", 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:42.349391+0000", 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 26.671419376999999, 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015226195, 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349391+0000", 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.109 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349391+0000", 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349392+0000", 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4600000000000004e-07 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.349418+0000", 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6214e-05 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.110 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359149+0000", 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0097303960000000005 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359157+0000", 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5350000000000004e-06 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359214+0000", 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6835e-05 2024-09-17T17:31:09.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.359223+0000", 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2019999999999993e-06 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.364582+0000", 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0053591580000000002 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.112 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.364617+0000", 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5108999999999999e-05 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.11.23:5 3.ds0 3:b7c231e8:::smithi10238019-17:head [assert-version v75,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.364515+0000", 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 20.656295802999999, 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011496884000000001, 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.113 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364515+0000", 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364515+0000", 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.114 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364517+0000", 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0760000000000001e-06 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364535+0000", 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7861999999999999e-05 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364564+0000", 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9190999999999999e-05 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.115 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364574+0000", 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.6409999999999997e-06 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364643+0000", 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.9018999999999998e-05 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364650+0000", 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5100000000000001e-06 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.367455+0000", 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0028047660000000002 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.367555+0000", 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000100485 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.376012+0000", 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0084563339999999994 2024-09-17T17:31:09.117 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:967 3.ds0 3:bc68f0c8:::smithi10238019-15:17 [read 0~3686400,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:51.753394+0000", 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.267416635, 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.016423462, 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.118 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 967 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753394+0000", 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753394+0000", 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.119 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753395+0000", 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3099999999999996e-07 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753400+0000", 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.8019999999999999e-06 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753409+0000", 2024-09-17T17:31:09.120 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4769999999999992e-06 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753415+0000", 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8100000000000003e-06 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.754122+0000", 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00070740199999999999 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.121 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.754222+0000", 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9247999999999996e-05 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.769817+0000", 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015595892 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:968 3.ds0 3:bc68f0c8:::smithi10238019-15:17 [read 1945998~1740402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:51.753438+0000", 2024-09-17T17:31:09.122 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 17.267372390999999, 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.017334796999999999, 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 968 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.123 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753438+0000", 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753438+0000", 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753439+0000", 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2799999999999996e-07 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.124 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753443+0000", 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.681e-06 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753451+0000", 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.0320000000000003e-06 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.753457+0000", 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0050000000000001e-06 2024-09-17T17:31:09.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.757035+0000", 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0035783719999999998 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.757169+0000", 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013432599999999999 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:51.770773+0000", 2024-09-17T17:31:09.126 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013603753 2024-09-17T17:31:09.127 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.127 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.127 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.127 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.127 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:09.128 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-17T17:31:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.231 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:09.231 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:31:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:09.240 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-17T17:31:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.342 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:09.342 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:31:09.342 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:31:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:09.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-17T17:31:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=24, last_complete=36'10, committed=1, applied=1))", 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 80.996430353999997, 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53983539199999997, 2024-09-17T17:31:09.454 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454670+0000", 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.454674+0000", 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0790000000000002e-06 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.456 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992637+0000", 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.53796236600000003 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.992644+0000", 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5820000000000004e-06 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994469+0000", 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0018243350000000001 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.457 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994505+0000", 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6300999999999997e-05 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:174 3.cs0 3:30756a83:::smithi10238019-35:2 [read 501714~71726,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 80.460962261000006, 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011875784, 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.458 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 174 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.459 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990138+0000", 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9400000000000005e-07 2024-09-17T17:31:09.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990142+0000", 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.557e-06 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990195+0000", 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3012e-05 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.461 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990206+0000", 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0876000000000001e-05 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990783+0000", 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00057770200000000003 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990850+0000", 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6168999999999996e-05 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.462 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.002014+0000", 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011163874000000001 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:175 3.cs0 3:30756a83:::smithi10238019-35:2 [read 194830~378610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 80.460875702999999, 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.30022850299999998, 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.463 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 175 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.464 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990224+0000", 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990225+0000", 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.8199999999999998e-07 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990230+0000", 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7809999999999996e-06 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990371+0000", 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00014071299999999999 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.990381+0000", 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0061000000000001e-05 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994702+0000", 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043214899999999999 2024-09-17T17:31:09.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:48.994769+0000", 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6600999999999995e-05 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290453+0000", 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.29568397499999999 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.467 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpReadReply(3.cs0 37/32 ECSubReadReply(tid=28, attrs_read=0))", 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 80.449902687000005, 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.289267987, 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.468 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001197+0000", 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001198+0000", 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6000000000000004e-07 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.469 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.001279+0000", 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1168000000000006e-05 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289843+0000", 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.28856357599999999 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289854+0000", 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1749999999999999e-05 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.470 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.289975+0000", 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00012060100000000001 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.290465+0000", 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049033200000000001 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.471 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:316 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (56) in=7b] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 74.461641921999998, 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041035350999999998, 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 316 2024-09-17T17:31:09.472 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989458+0000", 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.473 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989459+0000", 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2200000000000004e-07 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989462+0000", 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.287e-06 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989469+0000", 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7229999999999998e-06 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.474 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989472+0000", 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.428e-06 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990430+0000", 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00095784099999999997 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990525+0000", 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5129000000000003e-05 2024-09-17T17:31:09.475 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.990951+0000", 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00042552399999999999 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991309+0000", 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000358632 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.476 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030441+0000", 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039131753999999998 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030493+0000", 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2510999999999999e-05 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.477 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:317 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 74.461608585999997, 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.041694146000000001, 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 317 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.478 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989491+0000", 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989492+0000", 2024-09-17T17:31:09.479 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.3099999999999998e-07 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989494+0000", 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3329999999999999e-06 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989501+0000", 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.516e-06 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.480 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.989505+0000", 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.912e-06 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991167+0000", 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001662188 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:54.991214+0000", 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.6944000000000003e-05 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.481 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030733+0000", 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.039519406999999999 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.030782+0000", 2024-09-17T17:31:09.482 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.9338000000000001e-05 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:55.031186+0000", 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000403077 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.484 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:429 3.cs0 3:30d62bba:::smithi10238019-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.058197055999997, 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014137011, 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 429 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.485 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.486 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392903+0000", 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3299999999999998e-07 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392905+0000", 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6869999999999999e-06 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392912+0000", 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6250000000000001e-06 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.392916+0000", 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7330000000000003e-06 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397110+0000", 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0041939450000000001 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397209+0000", 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8961000000000003e-05 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.488 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402765+0000", 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0055556420000000004 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402840+0000", 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.4943000000000004e-05 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.407040+0000", 2024-09-17T17:31:09.489 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0042001419999999996 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:436 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 12] snapc b=[b,a,9,6] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.058100158000002, 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012854290000000001, 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.490 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 436 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393000+0000", 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3700000000000001e-07 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393005+0000", 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4190000000000004e-06 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.492 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393008+0000", 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7009999999999998e-06 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.393023+0000", 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4545999999999999e-05 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397903+0000", 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0048805719999999997 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.493 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.397964+0000", 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0520000000000003e-05 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.401036+0000", 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030720470000000001 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.402722+0000", 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001685513 2024-09-17T17:31:09.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405761+0000", 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0030398790000000001 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.405854+0000", 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2756000000000004e-05 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:486 3.fs0 3:f88f0a4d:::smithi10238019-16:head [setxattr _header (57) in=7b] snapc c=[a,9] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 62.072093664999997, 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75510421100000003, 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.496 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 486 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379006+0000", 2024-09-17T17:31:09.497 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379007+0000", 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900000000000003e-07 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379011+0000", 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.0720000000000001e-06 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.498 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379019+0000", 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9580000000000007e-06 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379025+0000", 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6799999999999998e-06 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379992+0000", 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00096721299999999997 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.499 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380044+0000", 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2091e-05 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380358+0000", 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000313543 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.500 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380731+0000", 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00037292 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134034+0000", 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75330350700000004 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134111+0000", 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6497999999999998e-05 2024-09-17T17:31:09.501 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:487 3.fs0 3:f88f0a4d:::smithi10238019-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 62.072064523999998, 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.756014517, 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.502 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 487 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379035+0000", 2024-09-17T17:31:09.503 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379036+0000", 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5499999999999998e-07 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379038+0000", 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2019999999999998e-06 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.504 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379045+0000", 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.8770000000000004e-06 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.379050+0000", 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2299999999999999e-06 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380522+0000", 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001471909 2024-09-17T17:31:09.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:07.380575+0000", 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2821999999999997e-05 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134417+0000", 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.75384171300000002 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.506 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.134482+0000", 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5789000000000003e-05 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:08.135050+0000", 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00056751999999999996 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:626 3.cs0 3:30756a83:::smithi10238019-35:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:09.507 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.952802687000002, 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021079339999999998, 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 626 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.508 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498297+0000", 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498298+0000", 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8299999999999997e-07 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498301+0000", 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7530000000000002e-06 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498307+0000", 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9869999999999996e-06 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498310+0000", 2024-09-17T17:31:09.510 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1259999999999998e-06 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499848+0000", 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001537912 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.499903+0000", 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5763000000000001e-05 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.511 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500222+0000", 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00031825499999999998 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500848+0000", 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062612400000000004 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519301+0000", 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018453431999999999 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.512 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519377+0000", 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5405000000000001e-05 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:627 3.cs0 3:30756a83:::smithi10238019-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.952773526999998, 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.021540693, 2024-09-17T17:31:09.513 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 627 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.514 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498326+0000", 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498327+0000", 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.51e-07 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498329+0000", 2024-09-17T17:31:09.515 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3989999999999998e-06 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498337+0000", 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5379999999999997e-06 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.498341+0000", 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4349999999999999e-06 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.516 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500569+0000", 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002227934 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500629+0000", 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9721e-05 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519409+0000", 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.018780192000000001 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.517 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519465+0000", 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6062e-05 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.519867+0000", 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000401961 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.518 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s6 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.950487222, 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26792300099999999, 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.519 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500613+0000", 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500614+0000", 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6700000000000002e-07 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.520 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500747+0000", 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00013382499999999999 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767249+0000", 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.26650138800000001 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767261+0000", 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1785e-05 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.521 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767372+0000", 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000111743 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.767387+0000", 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4548e-05 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768494+0000", 2024-09-17T17:31:09.522 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0011074990000000001 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.768536+0000", 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1346e-05 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:639 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [setxattr _header (57) in=7b] snapc 13=[13,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:09.523 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.927305216000001, 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67002614400000005, 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 639 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.524 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523795+0000", 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.0500000000000003e-07 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.525 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523799+0000", 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1729999999999999e-06 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523810+0000", 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1627000000000001e-05 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.526 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523815+0000", 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.0420000000000002e-06 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525161+0000", 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0013460169999999999 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.525253+0000", 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.1793999999999996e-05 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.527 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.526004+0000", 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00075112400000000004 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527034+0000", 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001029713 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193749+0000", 2024-09-17T17:31:09.528 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.66671533999999999 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.193821+0000", 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1709000000000001e-05 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:640 3.cs0 3:3ffd1ff3:::smithi10238019-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-17T17:31:09.529 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.927273599999999, 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.67078859899999999, 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 640 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.530 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523826+0000", 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523827+0000", 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.03e-07 2024-09-17T17:31:09.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523829+0000", 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1739999999999999e-06 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523836+0000", 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3869999999999997e-06 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.523841+0000", 2024-09-17T17:31:09.532 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4089999999999996e-06 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527242+0000", 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003401383 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for rw locks", 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.527298+0000", 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5705000000000003e-05 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.533 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194159+0000", 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.666861484 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194218+0000", 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9039000000000002e-05 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:20.194615+0000", 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00039661500000000001 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.534 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 68/32 ECSubWrite(tid=118, reqid=client.4566.0:641, at_version=68'91, trim_to=0'0, roll_forward_to=66'89))", 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 49.926432484000003, 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.164368816, 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.535 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524668+0000", 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.2899999999999999e-07 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.536 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.524694+0000", 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6256000000000001e-05 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687716+0000", 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.163021575 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687725+0000", 2024-09-17T17:31:09.537 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9390000000000003e-06 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687798+0000", 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2900999999999999e-05 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.687809+0000", 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.153e-05 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.538 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689007+0000", 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001197844 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.689036+0000", 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.9342000000000001e-05 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as6 78/32 ECSubWrite(tid=154, reqid=osd.3.0:154, at_version=78'120, trim_to=0'0, roll_forward_to=78'118))", 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:31:09.539 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.194715133999999, 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.088487576999999998, 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256385+0000", 2024-09-17T17:31:09.540 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256386+0000", 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3900000000000004e-07 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.256409+0000", 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3475000000000001e-05 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.541 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342362+0000", 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.085952671999999994 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.342372+0000", 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0285000000000001e-05 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343593+0000", 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001220858 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.542 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.343604+0000", 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1078e-05 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344835+0000", 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0012308309999999999 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.344872+0000", 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.7539000000000001e-05 2024-09-17T17:31:09.543 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 91/87 ECSubWrite(tid=84, reqid=osd.14.0:84, at_version=91'76, trim_to=0'0, roll_forward_to=89'72))", 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 27.097073318, 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.14743911800000001, 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.544 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354027+0000", 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:31:09.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.354048+0000", 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0214999999999999e-05 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500785+0000", 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.146736905 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.546 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500799+0000", 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4887e-05 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500878+0000", 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.8880000000000004e-05 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.500886+0000", 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.5270000000000001e-06 2024-09-17T17:31:09.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.501437+0000", 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000551134 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:42.501466+0000", 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.8836e-05 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.548 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:941 3.cs0 3:349443fd:::smithi10238019-43:head [copy-from ver 75] snapc 21=[21,20,18,17,16,15,10] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.089025136, 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.020459694, 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 941 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.549 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362075+0000", 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.550 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362077+0000", 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.984e-06 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362089+0000", 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2456000000000001e-05 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.551 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362107+0000", 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8053000000000001e-05 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362119+0000", 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2055999999999999e-05 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362183+0000", 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3246999999999999e-05 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.552 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362190+0000", 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.2470000000000002e-06 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364142+0000", 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0019523780000000001 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364249+0000", 2024-09-17T17:31:09.553 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010689899999999999 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.377910+0000", 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.013660960999999999 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.380133+0000", 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0022232409999999999 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.382430+0000", 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00229707 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.382535+0000", 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000104102 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:942 3.cs0 3:349443fd:::smithi10238019-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-17T17:31:09.555 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 21.088956502999999, 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.017198940999999999, 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 942 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.556 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362144+0000", 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.96e-07 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.557 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362147+0000", 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.5219999999999999e-06 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362153+0000", 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8159999999999999e-06 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362157+0000", 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.245e-06 2024-09-17T17:31:09.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362251+0000", 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3163000000000002e-05 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.362255+0000", 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1849999999999997e-06 2024-09-17T17:31:09.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364403+0000", 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002147726 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting for blocked object", 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.364448+0000", 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.5776999999999999e-05 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.560 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379154+0000", 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.014705371 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379256+0000", 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000102272 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:48.379342+0000", 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6468e-05 2024-09-17T17:31:09.561 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.562 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.562 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.562 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.562 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:09.563 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-17T17:31:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.666 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:09.666 INFO:teuthology.orchestra.run.smithi102.stdout: "num_ops": 0 2024-09-17T17:31:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:09.675 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-17T17:31:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.778 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [], 2024-09-17T17:31:09.778 INFO:teuthology.orchestra.run.smithi102.stdout: "complaint_time": 30, 2024-09-17T17:31:09.778 INFO:teuthology.orchestra.run.smithi102.stdout: "num_blocked_ops": 0 2024-09-17T17:31:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:09.786 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: "size": 20, 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 600, 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: "ops": [ 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 37/35 ECSubWriteReply(tid=17, last_complete=37'13, committed=1, applied=1))", 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:09.892 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 80.886744401000001, 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.068701408000000005, 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.893 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000933+0000", 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3399999999999998e-07 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.000937+0000", 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6449999999999998e-06 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.894 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068896+0000", 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.067959031000000003 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.068909+0000", 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3416e-05 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069008+0000", 2024-09-17T17:31:09.895 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.8690999999999994e-05 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:49.069634+0000", 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00062589099999999995 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.6s3 39/35 ECSubWrite(tid=47, reqid=client.4566.0:187, at_version=39'19, trim_to=0'0, roll_forward_to=36'17))", 2024-09-17T17:31:09.896 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 79.203813143000005, 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0079875469999999994, 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.897 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.683864+0000", 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2e-07 2024-09-17T17:31:09.898 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684336+0000", 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047134800000000001 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684439+0000", 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010291800000000001 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.899 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684449+0000", 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.027e-05 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684596+0000", 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000146749 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.900 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.684607+0000", 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1421e-05 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691787+0000", 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0071804069999999998 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.691851+0000", 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3913999999999998e-05 2024-09-17T17:31:09.901 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.as5 39/32 ECSubWrite(tid=28, reqid=osd.3.0:28, at_version=39'14, trim_to=0'0, roll_forward_to=37'12))", 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 79.202322737000003, 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089994210000000005, 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.902 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.903 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685354+0000", 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4100000000000001e-07 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.685373+0000", 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9063999999999999e-05 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687383+0000", 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.002009355 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.904 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687391+0000", 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.9699999999999999e-06 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687457+0000", 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5834999999999996e-05 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687467+0000", 2024-09-17T17:31:09.905 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.9990000000000003e-06 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694321+0000", 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0068546010000000001 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.694354+0000", 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2456000000000002e-05 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.906 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:191 3.5s0 3:a4d61510:::smithi10238019-32:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 3=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 79.200239874999994, 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.008961732, 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T17:31:09.907 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 191 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.908 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687437+0000", 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687439+0000", 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.1799999999999999e-06 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687504+0000", 2024-09-17T17:31:09.909 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.4770000000000005e-05 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687516+0000", 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2007e-05 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687531+0000", 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4803e-05 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687595+0000", 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3697999999999999e-05 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.687756+0000", 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000161846 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.690070+0000", 2024-09-17T17:31:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0023137639999999998 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695782+0000", 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0057116959999999996 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "commit_sent", 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.695908+0000", 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000126339 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.912 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:29:50.696399+0000", 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00049062900000000002 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:407 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.549599725999997, 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011701008000000001, 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.913 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 407 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.914 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338077+0000", 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338080+0000", 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2819999999999999e-06 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.915 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338088+0000", 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6289999999999999e-06 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338108+0000", 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.9514e-05 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338117+0000", 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.132e-06 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.916 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338188+0000", 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1446999999999995e-05 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338196+0000", 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6350000000000006e-06 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.341655+0000", 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0034593860000000001 2024-09-17T17:31:09.917 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342199+0000", 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00054402400000000005 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.349778+0000", 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075789589999999997 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:408 3.5s0 3:a96b502c:::smithi10238019-42:head [read 53220~651292,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.549437799000003, 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012064669, 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 408 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.919 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338239+0000", 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.920 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338240+0000", 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1699999999999997e-07 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338257+0000", 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6643e-05 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338270+0000", 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.3081000000000001e-05 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.921 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338277+0000", 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3780000000000003e-06 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338331+0000", 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.4286999999999997e-05 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338338+0000", 2024-09-17T17:31:09.922 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.3609999999999997e-06 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342681+0000", 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0043432749999999997 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.342775+0000", 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4392e-05 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.923 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.350304+0000", 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.007528435 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:409 3.5s0 3:a96b502c:::smithi10238019-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.549325044, 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012665852, 2024-09-17T17:31:09.924 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 409 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.925 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338352+0000", 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338353+0000", 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.09e-07 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338358+0000", 2024-09-17T17:31:09.926 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.7859999999999996e-06 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338366+0000", 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.6049999999999998e-06 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338372+0000", 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1739999999999997e-06 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.927 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338439+0000", 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7039999999999995e-05 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.338446+0000", 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.1160000000000001e-06 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.928 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343114+0000", 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0046672179999999999 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.343207+0000", 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.3170000000000004e-05 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.929 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.351018+0000", 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0078109340000000003 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.2s4 51/32 ECSubWrite(tid=26, reqid=osd.14.0:26, at_version=51'24, trim_to=0'0, roll_forward_to=48'22))", 2024-09-17T17:31:09.930 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 67.535170664000006, 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.988933536, 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.931 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352506+0000", 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352507+0000", 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1500000000000003e-07 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:02.352532+0000", 2024-09-17T17:31:09.932 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4485999999999999e-05 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340513+0000", 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.98798163100000003 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340522+0000", 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.9379999999999998e-06 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.933 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340625+0000", 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010273 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.340633+0000", 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1610000000000003e-06 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341398+0000", 2024-09-17T17:31:09.934 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00076522600000000004 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.341440+0000", 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.1548999999999998e-05 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/35 ECSubWrite(tid=50, reqid=client.4566.0:441, at_version=0'0, trim_to=0'0, roll_forward_to=52'37))", 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:31:09.935 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 66.024113143999998, 2024-09-17T17:31:09.936 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.121580543, 2024-09-17T17:31:09.936 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.936 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.936 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.936 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.936 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863564+0000", 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863565+0000", 2024-09-17T17:31:09.937 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6729999999999999e-06 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.863588+0000", 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.2756e-05 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984520+0000", 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.120931489 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.938 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984547+0000", 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6877999999999999e-05 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984631+0000", 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4246000000000006e-05 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.984638+0000", 2024-09-17T17:31:09.939 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3509999999999996e-06 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985117+0000", 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00047851700000000003 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:03.985144+0000", 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.7633000000000001e-05 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.940 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.12s4 68/67 ECSubWrite(tid=47, reqid=client.4566.0:628, at_version=68'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 50.387022610999999, 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.090576745, 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.941 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500654+0000", 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500656+0000", 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0430000000000002e-06 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.942 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.500772+0000", 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000115338 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590079+0000", 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.089307485000000006 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590103+0000", 2024-09-17T17:31:09.943 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.4131e-05 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590159+0000", 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5417000000000002e-05 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.590177+0000", 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7819e-05 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.944 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591201+0000", 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.001023939 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:19.591231+0000", 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0573e-05 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.945 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 75/74 ECSubWrite(tid=94, reqid=osd.10.0:94, at_version=75'58, trim_to=0'0, roll_forward_to=0'0))", 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 43.535554984999997, 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0086672569999999994, 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.946 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352122+0000", 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352123+0000", 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6199999999999997e-07 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.947 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352143+0000", 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0440000000000001e-05 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352170+0000", 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.6503e-05 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.948 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352177+0000", 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.6429999999999995e-06 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352236+0000", 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8711000000000001e-05 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.352245+0000", 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.0720000000000006e-06 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.949 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360755+0000", 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085095469999999993 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:26.360789+0000", 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4678999999999998e-05 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.950 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 78/74 ECSubWrite(tid=102, reqid=client.4566.0:771, at_version=78'64, trim_to=0'0, roll_forward_to=78'60))", 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 40.505823644000003, 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96796263999999999, 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.951 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381853+0000", 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381854+0000", 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.4e-07 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:29.381870+0000", 2024-09-17T17:31:09.952 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.6076e-05 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348683+0000", 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.96681319499999996 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348717+0000", 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.3704000000000001e-05 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.953 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348821+0000", 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010401800000000001 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.348833+0000", 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2172e-05 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349780+0000", 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00094660899999999995 2024-09-17T17:31:09.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.349816+0000", 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6025999999999997e-05 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 79/74 ECSubWrite(tid=108, reqid=client.4566.0:788, at_version=79'60, trim_to=0'0, roll_forward_to=79'56))", 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 39.522117592000001, 2024-09-17T17:31:09.955 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010065843999999999, 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.956 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365559+0000", 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365560+0000", 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5199999999999995e-07 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.365578+0000", 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.8145999999999999e-05 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.957 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374667+0000", 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.009088245 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374697+0000", 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.0402000000000001e-05 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374763+0000", 2024-09-17T17:31:09.958 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.5912000000000002e-05 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.374776+0000", 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.2497000000000001e-05 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375591+0000", 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00081505399999999995 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.959 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:30.375625+0000", 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.4635999999999998e-05 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.13s3 80/74 ECSubWrite(tid=112, reqid=osd.10.0:111, at_version=0'0, trim_to=0'0, roll_forward_to=80'62))", 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 38.535293760999998, 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0076437279999999998, 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.960 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352383+0000", 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.961 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352384+0000", 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0249999999999999e-06 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352400+0000", 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5628999999999999e-05 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352437+0000", 2024-09-17T17:31:09.962 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.6813000000000001e-05 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352448+0000", 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0882e-05 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352672+0000", 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00022395999999999999 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.963 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.352683+0000", 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1656e-05 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.359996+0000", 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0073125120000000002 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:31.360027+0000", 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.1251000000000003e-05 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.964 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(osd.13.22:9 3.5s0 3:a4d61510:::smithi10238019-32:head [assert-version v27,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 33.545586350999997, 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.015638470000000002, 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.965 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342091+0000", 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342092+0000", 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.7e-06 2024-09-17T17:31:09.966 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342098+0000", 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.8690000000000001e-06 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342114+0000", 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5954999999999999e-05 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342122+0000", 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3919999999999997e-06 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342179+0000", 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.7768e-05 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.342187+0000", 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3819999999999998e-06 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348515+0000", 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0063278780000000003 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.348672+0000", 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00015732800000000001 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.357729+0000", 2024-09-17T17:31:09.969 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0090571980000000007 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.9s4 85/76 ECSubWrite(tid=194, reqid=osd.12.0:193, at_version=0'0, trim_to=0'0, roll_forward_to=85'63))", 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 33.535373915000001, 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0091352759999999995, 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.970 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352303+0000", 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.971 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352304+0000", 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.4e-07 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352316+0000", 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1864e-05 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352360+0000", 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 4.4478999999999999e-05 2024-09-17T17:31:09.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352370+0000", 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.5829999999999996e-06 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352437+0000", 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.7521000000000006e-05 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352447+0000", 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2180000000000005e-06 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361372+0000", 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0089248839999999992 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361438+0000", 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.6786999999999996e-05 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.974 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "MOSDECSubOpWrite(3.3s3 85/74 ECSubWrite(tid=125, reqid=osd.10.0:125, at_version=85'76, trim_to=0'0, roll_forward_to=83'74))", 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 33.534739580999997, 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0085212919999999998, 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "reached pg", 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.975 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352937+0000", 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352938+0000", 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.04e-07 2024-09-17T17:31:09.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.352958+0000", 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.0486e-05 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353044+0000", 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.5852000000000004e-05 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.977 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353050+0000", 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2280000000000003e-06 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353110+0000", 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.9478999999999998e-05 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_started", 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.353121+0000", 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.1426e-05 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.978 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "sub_op_committed", 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361426+0000", 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0083047659999999999 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:36.361459+0000", 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 3.2851000000000001e-05 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:922 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.542163838, 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.010998030000000001, 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 922 2024-09-17T17:31:09.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345513+0000", 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.981 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345515+0000", 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.4219999999999999e-06 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345520+0000", 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2909999999999998e-06 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345543+0000", 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 2.3323e-05 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345551+0000", 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 7.3549999999999999e-06 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345653+0000", 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.00010261500000000001 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "op must wait for map", 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345661+0000", 2024-09-17T17:31:09.983 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.1310000000000006e-06 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.352634+0000", 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0069729299999999996 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.352757+0000", 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000122714 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.984 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.356511+0000", 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0037542489999999999 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:923 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.542109462999999, 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.011747277, 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.985 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 923 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.986 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345567+0000", 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345568+0000", 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.6300000000000005e-07 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.987 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345574+0000", 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.5219999999999998e-06 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345580+0000", 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.1120000000000001e-06 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345585+0000", 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.3229999999999997e-06 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.988 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345814+0000", 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000228989 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345820+0000", 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 6.2419999999999997e-06 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.353324+0000", 2024-09-17T17:31:09.989 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0075038520000000001 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.353431+0000", 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.000107129 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.357315+0000", 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003883545 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.990 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "description": "osd_op(client.4566.0:924 3.15s0 3:ada165d5:::smithi10238019-13:15 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "initiated_at": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "age": 22.541948869999999, 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.012050959, 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "continuous": false, 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "type_data": { 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "flag_point": "started", 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "client_info": { 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "client": "client.4566", 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "client_addr": "172.21.15.102:0/2799856256", 2024-09-17T17:31:09.991 INFO:teuthology.orchestra.run.smithi102.stdout: "tid": 924 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "events": [ 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "initiated", 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "header_read", 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345728+0000", 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.992 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "throttled", 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345729+0000", 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.015e-06 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "all_read", 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345734+0000", 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2909999999999998e-06 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "dispatched", 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345745+0000", 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.0134e-05 2024-09-17T17:31:09.993 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "queued_for_pg", 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345760+0000", 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 1.5775999999999999e-05 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345844+0000", 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 8.3635999999999997e-05 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "waiting_for_map not empty", 2024-09-17T17:31:09.994 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.345849+0000", 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 5.2100000000000001e-06 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "reached_pg", 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.354091+0000", 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.0082413780000000006 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "started", 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.354183+0000", 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 9.2315e-05 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-17T17:31:09.995 INFO:teuthology.orchestra.run.smithi102.stdout: "event": "done", 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: "time": "2024-09-17T17:30:47.357779+0000", 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: "duration": 0.003596204 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-17T17:31:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-17T17:31:13.255 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-17T17:31:13.255 INFO:tasks.thrashosds.thrasher:Reviving osd 8 2024-09-17T17:31:13.255 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-17T17:31:13.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-17T17:31:13.259 INFO:tasks.ceph.osd.8:Started 2024-09-17T17:31:13.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:31:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:config diff 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:config diff get 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:config get 2024-09-17T17:31:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:config help [] 2024-09-17T17:31:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:config set ... 2024-09-17T17:31:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:admin_socket: invalid command 2024-09-17T17:31:13.375 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T17:31:13.375 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-8, ['dump_ops_in_flight'] 2024-09-17T17:31:14.442 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:31:14.440+0000 7f2f8a543640 -1 Falling back to public interface 2024-09-17T17:31:18.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:31:18.488 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:31:18.486+0000 7f2f8a543640 -1 osd.8 110 log_to_monitors true 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T17:31:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T17:31:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T17:31:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T17:31:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator fragmentation block 2024-09-17T17:31:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:admin_socket: invalid command 2024-09-17T17:31:18.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T17:31:18.496 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-8, ['dump_ops_in_flight'] 2024-09-17T17:31:18.980 INFO:tasks.ceph.osd.8.smithi007.stderr:2024-09-17T17:31:18.977+0000 7f2f7e2f1640 -1 osd.8 116 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T17:31:23.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-17T17:31:23.617 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:31:23.617 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T17:31:23.618 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T17:31:23.618 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:31:23.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T17:31:23.752 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-17T17:31:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T17:31:23.767 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-17T17:31:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T17:31:23.781 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-17T17:31:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T17:31:23.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T17:31:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T17:31:25.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T17:31:25.988 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T17:31:26.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T17:31:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T17:31:27.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T17:31:28.006 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T17:31:28.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-17T17:31:29.012 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.12 to 1 (10000) 2024-09-17T17:31:29.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-09-17T17:31:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-09-17T17:31:30.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T17:31:31.023 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T17:31:31.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T17:31:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T17:31:32.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T17:31:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T17:31:33.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T17:31:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T17:31:34.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-17T17:31:35.048 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.9 to 1 (10000) 2024-09-17T17:31:35.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-17T17:31:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-17T17:31:36.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-17T17:31:37.061 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.13 to 1 (10000) 2024-09-17T17:31:37.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-09-17T17:31:38.072 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-09-17T17:31:38.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T17:31:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T17:31:39.091 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T17:31:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T17:31:40.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T17:31:41.083 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T17:31:41.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-17T17:31:42.089 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-17T17:31:42.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-17T17:31:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.10 to 1 (10000) 2024-09-17T17:31:43.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-17T17:31:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-17T17:31:44.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-17T17:31:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.14 to 1 (10000) 2024-09-17T17:31:45.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-09-17T17:31:46.118 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-09-17T17:31:46.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T17:31:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T17:31:47.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T17:31:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T17:31:48.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T17:31:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T17:31:49.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-17T17:31:50.145 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-17T17:31:50.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-17T17:31:51.147 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.11 to 1 (10000) 2024-09-17T17:31:51.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-17T17:31:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-17T17:31:52.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-17T17:31:53.157 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.15 to 1 (10000) 2024-09-17T17:31:53.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-09-17T17:31:54.162 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-09-17T17:31:54.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-17T17:31:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.8 to 1 (10000) 2024-09-17T17:31:55.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-17T17:31:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-17T17:31:56.196 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T17:31:56.196 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T17:31:56.196 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T17:31:56.196 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T17:31:56.196 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T17:31:56.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:31:56.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:31:56.512+0000 7f657ed78640 0 --2- 172.21.15.7:0/641686024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f657815acb0 0x7f657814d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T17:31:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:31:56.713 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":150,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:31:56.172259+0000","last_up_change":"2024-09-17T17:31:19.959931+0000","last_in_change":"2024-09-17T17:30:22.332815+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T17:29:30.150043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":23,"last_clean_end":110,"up_from":117,"up_thru":117,"down_at":112,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6809","nonce":3844390318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6811","nonce":3844390318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6815","nonce":3844390318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6813","nonce":3844390318}]},"public_addr":"172.21.15.7:6809/3844390318","cluster_addr":"172.21.15.7:6811/3844390318","heartbeat_back_addr":"172.21.15.7:6815/3844390318","heartbeat_front_addr":"172.21.15.7:6813/3844390318","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:53.149089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:29.488817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:53.727934+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:44.107535+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:43.930932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:46.730260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:48.564572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:46.687618+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:46.536431+0000","dead_epoch":112},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:44.042967+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:33.448841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:41.027274+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:33.749926+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.511757+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.711104+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:48.758623+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:31:56.725 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T17:31:56.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:31:57.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:31:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":150,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:31:56.172259+0000","last_up_change":"2024-09-17T17:31:19.959931+0000","last_in_change":"2024-09-17T17:30:22.332815+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T17:29:30.150043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":23,"last_clean_end":110,"up_from":117,"up_thru":117,"down_at":112,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6809","nonce":3844390318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6811","nonce":3844390318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6815","nonce":3844390318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6813","nonce":3844390318}]},"public_addr":"172.21.15.7:6809/3844390318","cluster_addr":"172.21.15.7:6811/3844390318","heartbeat_back_addr":"172.21.15.7:6815/3844390318","heartbeat_front_addr":"172.21.15.7:6813/3844390318","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:53.149089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:29.488817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:53.727934+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:44.107535+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:43.930932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:46.730260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:48.564572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:46.687618+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:46.536431+0000","dead_epoch":112},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:44.042967+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:33.448841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:41.027274+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:33.749926+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.511757+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.711104+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:48.758623+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:31:57.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T17:31:57.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T17:31:57.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T17:31:57.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T17:31:57.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T17:31:57.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T17:31:57.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T17:31:57.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T17:31:57.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-17T17:31:57.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-17T17:31:57.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-17T17:31:57.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-17T17:31:57.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-17T17:31:57.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-17T17:31:57.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-17T17:31:57.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-17T17:31:57.457 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:57.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-17T17:31:57.484 INFO:teuthology.orchestra.run.smithi007.stdout:98784247841 2024-09-17T17:31:57.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-17T17:31:57.497 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:57.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T17:31:57.514 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:57.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T17:31:57.520 INFO:teuthology.orchestra.run.smithi007.stdout:502511173641 2024-09-17T17:31:57.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-17T17:31:57.543 INFO:teuthology.orchestra.run.smithi007.stdout:98784247841 2024-09-17T17:31:57.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T17:31:57.547 INFO:teuthology.orchestra.run.smithi007.stdout:90194313250 2024-09-17T17:31:57.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T17:31:57.555 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:57.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T17:31:57.564 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:57.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T17:31:57.565 INFO:teuthology.orchestra.run.smithi007.stdout:90194313250 2024-09-17T17:31:57.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T17:31:57.574 INFO:teuthology.orchestra.run.smithi007.stdout:98784247842 2024-09-17T17:31:57.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T17:31:57.586 INFO:teuthology.orchestra.run.smithi007.stdout:98784247841 2024-09-17T17:31:57.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-17T17:31:57.599 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:57.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-17T17:31:57.614 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:57.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-17T17:31:57.631 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:57.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-17T17:31:57.683 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:57.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-17T17:31:58.499 INFO:teuthology.orchestra.run.smithi007.stdout:98784247841 2024-09-17T17:31:58.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247841 got 98784247841 for osd.12 2024-09-17T17:31:58.539 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.575 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:58.586 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:58.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280546 got 94489280546 for osd.0 2024-09-17T17:31:58.616 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280545 got 94489280545 for osd.2 2024-09-17T17:31:58.633 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.733 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:58.733 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:58.736 INFO:teuthology.orchestra.run.smithi007.stdout:90194313250 2024-09-17T17:31:58.749 INFO:teuthology.orchestra.run.smithi007.stdout:98784247842 2024-09-17T17:31:58.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280546 got 94489280546 for osd.9 2024-09-17T17:31:58.767 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280545 got 94489280545 for osd.15 2024-09-17T17:31:58.770 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313250 got 90194313250 for osd.1 2024-09-17T17:31:58.784 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.784 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:58.795 INFO:teuthology.orchestra.run.smithi007.stdout:94489280546 2024-09-17T17:31:58.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247842 got 98784247842 for osd.4 2024-09-17T17:31:58.799 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.800 INFO:teuthology.orchestra.run.smithi007.stdout:502511173641 2024-09-17T17:31:58.811 INFO:teuthology.orchestra.run.smithi007.stdout:98784247841 2024-09-17T17:31:58.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280546 got 94489280546 for osd.3 2024-09-17T17:31:58.819 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280546 got 94489280546 for osd.5 2024-09-17T17:31:58.820 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.834 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:58.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 502511173641 got 502511173641 for osd.8 2024-09-17T17:31:58.835 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247841 got 98784247841 for osd.11 2024-09-17T17:31:58.836 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.838 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:58.850 INFO:teuthology.orchestra.run.smithi007.stdout:98784247841 2024-09-17T17:31:58.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280545 got 94489280545 for osd.13 2024-09-17T17:31:58.852 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.855 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280545 got 94489280545 for osd.14 2024-09-17T17:31:58.855 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.856 INFO:teuthology.orchestra.run.smithi007.stdout:90194313251 2024-09-17T17:31:58.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247841 got 98784247841 for osd.6 2024-09-17T17:31:58.865 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313250 got 90194313251 for osd.7 2024-09-17T17:31:58.870 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.881 INFO:teuthology.orchestra.run.smithi007.stdout:94489280545 2024-09-17T17:31:58.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280545 got 94489280545 for osd.10 2024-09-17T17:31:58.896 DEBUG:teuthology.parallel:result is None 2024-09-17T17:31:58.896 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T17:31:58.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:31:59.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:31:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:31:59.252 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-17T17:31:57.900477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":612564,"kb_used_data":37000,"kb_used_omap":33,"kb_used_meta":575134,"kb_avail":1509336876,"statfs":{"total":1546188226560,"available":1545560961024,"internally_reserved":0,"allocated":37888000,"data_stored":22713236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":588938099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.743831"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182777+0000","last_change":"2024-09-17T17:31:49.863875+0000","last_active":"2024-09-17T17:31:56.182777+0000","last_peered":"2024-09-17T17:31:56.182777+0000","last_clean":"2024-09-17T17:31:56.182777+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:31:56.182777+0000","last_undegraded":"2024-09-17T17:31:56.182777+0000","last_fullsized":"2024-09-17T17:31:56.182777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:49.863800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.863800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:53.247141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":269,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182878+0000","last_change":"2024-09-17T17:31:34.252433+0000","last_active":"2024-09-17T17:31:56.182878+0000","last_peered":"2024-09-17T17:31:56.182878+0000","last_clean":"2024-09-17T17:31:56.182878+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:31:56.182878+0000","last_undegraded":"2024-09-17T17:31:56.182878+0000","last_fullsized":"2024-09-17T17:31:56.182878+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:34.252354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:34.252354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:01.572165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.834621+0000","last_change":"2024-09-17T17:31:22.541193+0000","last_active":"2024-09-17T17:31:56.834621+0000","last_peered":"2024-09-17T17:31:56.834621+0000","last_clean":"2024-09-17T17:31:56.834621+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:31:56.834621+0000","last_undegraded":"2024-09-17T17:31:56.834621+0000","last_fullsized":"2024-09-17T17:31:56.834621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:22.541119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:22.541119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:24.275934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":272,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.221669+0000","last_change":"2024-09-17T17:31:50.178015+0000","last_active":"2024-09-17T17:31:56.221669+0000","last_peered":"2024-09-17T17:31:56.221669+0000","last_clean":"2024-09-17T17:31:56.221669+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:31:56.221669+0000","last_undegraded":"2024-09-17T17:31:56.221669+0000","last_fullsized":"2024-09-17T17:31:56.221669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:50.177917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:50.177917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:50.893244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178596+0000","last_change":"2024-09-17T17:31:56.070122+0000","last_active":"2024-09-17T17:31:56.178596+0000","last_peered":"2024-09-17T17:31:56.178596+0000","last_clean":"2024-09-17T17:31:56.178596+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:31:56.178596+0000","last_undegraded":"2024-09-17T17:31:56.178596+0000","last_fullsized":"2024-09-17T17:31:56.178596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-17T17:31:56.070038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:56.070038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.017596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183107+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:31:56.183107+0000","last_peered":"2024-09-17T17:31:56.183107+0000","last_clean":"2024-09-17T17:31:56.183107+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:31:56.183107+0000","last_undegraded":"2024-09-17T17:31:56.183107+0000","last_fullsized":"2024-09-17T17:31:56.183107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178594+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:31:56.178594+0000","last_peered":"2024-09-17T17:31:56.178594+0000","last_clean":"2024-09-17T17:31:56.178594+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:31:56.178594+0000","last_undegraded":"2024-09-17T17:31:56.178594+0000","last_fullsized":"2024-09-17T17:31:56.178594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183111+0000","last_change":"2024-09-17T17:31:49.471816+0000","last_active":"2024-09-17T17:31:56.183111+0000","last_peered":"2024-09-17T17:31:56.183111+0000","last_clean":"2024-09-17T17:31:56.183111+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:31:56.183111+0000","last_undegraded":"2024-09-17T17:31:56.183111+0000","last_fullsized":"2024-09-17T17:31:56.183111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:31:49.471731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.471731+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:54.889153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":404,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.254177+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:31:56.254177+0000","last_peered":"2024-09-17T17:31:56.254177+0000","last_clean":"2024-09-17T17:31:56.254177+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:31:56.254177+0000","last_undegraded":"2024-09-17T17:31:56.254177+0000","last_fullsized":"2024-09-17T17:31:56.254177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173641,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30800,"kb_used_data":2232,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94341040,"statfs":{"total":96636764160,"available":96605224960,"internally_reserved":0,"allocated":2285568,"data_stored":1344472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33968,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337872,"statfs":{"total":96636764160,"available":96601980928,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37604,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334236,"statfs":{"total":96636764160,"available":96598257664,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37608,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334232,"statfs":{"total":96636764160,"available":96598253568,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247841,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38116,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:31:59.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:31:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:31:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:31:59.571 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-17T17:31:57.900477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":612564,"kb_used_data":37000,"kb_used_omap":33,"kb_used_meta":575134,"kb_avail":1509336876,"statfs":{"total":1546188226560,"available":1545560961024,"internally_reserved":0,"allocated":37888000,"data_stored":22713236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":588938099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.743831"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182777+0000","last_change":"2024-09-17T17:31:49.863875+0000","last_active":"2024-09-17T17:31:56.182777+0000","last_peered":"2024-09-17T17:31:56.182777+0000","last_clean":"2024-09-17T17:31:56.182777+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:31:56.182777+0000","last_undegraded":"2024-09-17T17:31:56.182777+0000","last_fullsized":"2024-09-17T17:31:56.182777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:49.863800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.863800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:53.247141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":269,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182878+0000","last_change":"2024-09-17T17:31:34.252433+0000","last_active":"2024-09-17T17:31:56.182878+0000","last_peered":"2024-09-17T17:31:56.182878+0000","last_clean":"2024-09-17T17:31:56.182878+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:31:56.182878+0000","last_undegraded":"2024-09-17T17:31:56.182878+0000","last_fullsized":"2024-09-17T17:31:56.182878+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:34.252354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:34.252354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:01.572165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.834621+0000","last_change":"2024-09-17T17:31:22.541193+0000","last_active":"2024-09-17T17:31:56.834621+0000","last_peered":"2024-09-17T17:31:56.834621+0000","last_clean":"2024-09-17T17:31:56.834621+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:31:56.834621+0000","last_undegraded":"2024-09-17T17:31:56.834621+0000","last_fullsized":"2024-09-17T17:31:56.834621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:22.541119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:22.541119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:24.275934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":272,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.221669+0000","last_change":"2024-09-17T17:31:50.178015+0000","last_active":"2024-09-17T17:31:56.221669+0000","last_peered":"2024-09-17T17:31:56.221669+0000","last_clean":"2024-09-17T17:31:56.221669+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:31:56.221669+0000","last_undegraded":"2024-09-17T17:31:56.221669+0000","last_fullsized":"2024-09-17T17:31:56.221669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:50.177917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:50.177917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:50.893244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178596+0000","last_change":"2024-09-17T17:31:56.070122+0000","last_active":"2024-09-17T17:31:56.178596+0000","last_peered":"2024-09-17T17:31:56.178596+0000","last_clean":"2024-09-17T17:31:56.178596+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:31:56.178596+0000","last_undegraded":"2024-09-17T17:31:56.178596+0000","last_fullsized":"2024-09-17T17:31:56.178596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-17T17:31:56.070038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:56.070038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.017596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183107+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:31:56.183107+0000","last_peered":"2024-09-17T17:31:56.183107+0000","last_clean":"2024-09-17T17:31:56.183107+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:31:56.183107+0000","last_undegraded":"2024-09-17T17:31:56.183107+0000","last_fullsized":"2024-09-17T17:31:56.183107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178594+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:31:56.178594+0000","last_peered":"2024-09-17T17:31:56.178594+0000","last_clean":"2024-09-17T17:31:56.178594+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:31:56.178594+0000","last_undegraded":"2024-09-17T17:31:56.178594+0000","last_fullsized":"2024-09-17T17:31:56.178594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183111+0000","last_change":"2024-09-17T17:31:49.471816+0000","last_active":"2024-09-17T17:31:56.183111+0000","last_peered":"2024-09-17T17:31:56.183111+0000","last_clean":"2024-09-17T17:31:56.183111+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:31:56.183111+0000","last_undegraded":"2024-09-17T17:31:56.183111+0000","last_fullsized":"2024-09-17T17:31:56.183111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:31:49.471731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.471731+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:54.889153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":404,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.254177+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:31:56.254177+0000","last_peered":"2024-09-17T17:31:56.254177+0000","last_clean":"2024-09-17T17:31:56.254177+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:31:56.254177+0000","last_undegraded":"2024-09-17T17:31:56.254177+0000","last_fullsized":"2024-09-17T17:31:56.254177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173641,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30800,"kb_used_data":2232,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94341040,"statfs":{"total":96636764160,"available":96605224960,"internally_reserved":0,"allocated":2285568,"data_stored":1344472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33968,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337872,"statfs":{"total":96636764160,"available":96601980928,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37604,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334236,"statfs":{"total":96636764160,"available":96598257664,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37608,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334232,"statfs":{"total":96636764160,"available":96598253568,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247841,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38116,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:31:59.572 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T17:31:59.572 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T17:31:59.586 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T17:31:59.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:31:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:31:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:31:59.900 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-17T17:31:57.900477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":612564,"kb_used_data":37000,"kb_used_omap":33,"kb_used_meta":575134,"kb_avail":1509336876,"statfs":{"total":1546188226560,"available":1545560961024,"internally_reserved":0,"allocated":37888000,"data_stored":22713236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":588938099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.743831"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182777+0000","last_change":"2024-09-17T17:31:49.863875+0000","last_active":"2024-09-17T17:31:56.182777+0000","last_peered":"2024-09-17T17:31:56.182777+0000","last_clean":"2024-09-17T17:31:56.182777+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:31:56.182777+0000","last_undegraded":"2024-09-17T17:31:56.182777+0000","last_fullsized":"2024-09-17T17:31:56.182777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:49.863800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.863800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:53.247141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":269,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182878+0000","last_change":"2024-09-17T17:31:34.252433+0000","last_active":"2024-09-17T17:31:56.182878+0000","last_peered":"2024-09-17T17:31:56.182878+0000","last_clean":"2024-09-17T17:31:56.182878+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:31:56.182878+0000","last_undegraded":"2024-09-17T17:31:56.182878+0000","last_fullsized":"2024-09-17T17:31:56.182878+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:34.252354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:34.252354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:01.572165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.834621+0000","last_change":"2024-09-17T17:31:22.541193+0000","last_active":"2024-09-17T17:31:56.834621+0000","last_peered":"2024-09-17T17:31:56.834621+0000","last_clean":"2024-09-17T17:31:56.834621+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:31:56.834621+0000","last_undegraded":"2024-09-17T17:31:56.834621+0000","last_fullsized":"2024-09-17T17:31:56.834621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:22.541119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:22.541119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:24.275934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":272,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.221669+0000","last_change":"2024-09-17T17:31:50.178015+0000","last_active":"2024-09-17T17:31:56.221669+0000","last_peered":"2024-09-17T17:31:56.221669+0000","last_clean":"2024-09-17T17:31:56.221669+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:31:56.221669+0000","last_undegraded":"2024-09-17T17:31:56.221669+0000","last_fullsized":"2024-09-17T17:31:56.221669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:50.177917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:50.177917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:50.893244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178596+0000","last_change":"2024-09-17T17:31:56.070122+0000","last_active":"2024-09-17T17:31:56.178596+0000","last_peered":"2024-09-17T17:31:56.178596+0000","last_clean":"2024-09-17T17:31:56.178596+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:31:56.178596+0000","last_undegraded":"2024-09-17T17:31:56.178596+0000","last_fullsized":"2024-09-17T17:31:56.178596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-17T17:31:56.070038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:56.070038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.017596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183107+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:31:56.183107+0000","last_peered":"2024-09-17T17:31:56.183107+0000","last_clean":"2024-09-17T17:31:56.183107+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:31:56.183107+0000","last_undegraded":"2024-09-17T17:31:56.183107+0000","last_fullsized":"2024-09-17T17:31:56.183107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178594+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:31:56.178594+0000","last_peered":"2024-09-17T17:31:56.178594+0000","last_clean":"2024-09-17T17:31:56.178594+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:31:56.178594+0000","last_undegraded":"2024-09-17T17:31:56.178594+0000","last_fullsized":"2024-09-17T17:31:56.178594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183111+0000","last_change":"2024-09-17T17:31:49.471816+0000","last_active":"2024-09-17T17:31:56.183111+0000","last_peered":"2024-09-17T17:31:56.183111+0000","last_clean":"2024-09-17T17:31:56.183111+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:31:56.183111+0000","last_undegraded":"2024-09-17T17:31:56.183111+0000","last_fullsized":"2024-09-17T17:31:56.183111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:31:49.471731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.471731+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:54.889153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":404,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.254177+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:31:56.254177+0000","last_peered":"2024-09-17T17:31:56.254177+0000","last_clean":"2024-09-17T17:31:56.254177+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:31:56.254177+0000","last_undegraded":"2024-09-17T17:31:56.254177+0000","last_fullsized":"2024-09-17T17:31:56.254177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173641,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30800,"kb_used_data":2232,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94341040,"statfs":{"total":96636764160,"available":96605224960,"internally_reserved":0,"allocated":2285568,"data_stored":1344472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33968,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337872,"statfs":{"total":96636764160,"available":96601980928,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37604,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334236,"statfs":{"total":96636764160,"available":96598257664,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37608,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334232,"statfs":{"total":96636764160,"available":96598253568,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247841,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38116,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:31:59.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:32:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T17:32:00.204+0000 7f303ffff640 0 --2- 172.21.15.7:0/3316132574 >> v2:172.21.15.26:3300/0 conn(0x7f304015bc40 0x7f304015c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T17:32:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:32:00.410 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:32:00.422 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-17T17:32:00.188281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":612584,"kb_used_data":37016,"kb_used_omap":33,"kb_used_meta":575134,"kb_avail":1509336856,"statfs":{"total":1546188226560,"available":1545560940544,"internally_reserved":0,"allocated":37904384,"data_stored":22722822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":588938099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.025741"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182777+0000","last_change":"2024-09-17T17:31:49.863875+0000","last_active":"2024-09-17T17:31:56.182777+0000","last_peered":"2024-09-17T17:31:56.182777+0000","last_clean":"2024-09-17T17:31:56.182777+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:31:56.182777+0000","last_undegraded":"2024-09-17T17:31:56.182777+0000","last_fullsized":"2024-09-17T17:31:56.182777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:49.863800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.863800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:53.247141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":269,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182878+0000","last_change":"2024-09-17T17:31:34.252433+0000","last_active":"2024-09-17T17:31:56.182878+0000","last_peered":"2024-09-17T17:31:56.182878+0000","last_clean":"2024-09-17T17:31:56.182878+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:31:56.182878+0000","last_undegraded":"2024-09-17T17:31:56.182878+0000","last_fullsized":"2024-09-17T17:31:56.182878+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:34.252354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:34.252354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:01.572165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.834621+0000","last_change":"2024-09-17T17:31:22.541193+0000","last_active":"2024-09-17T17:31:56.834621+0000","last_peered":"2024-09-17T17:31:56.834621+0000","last_clean":"2024-09-17T17:31:56.834621+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:31:56.834621+0000","last_undegraded":"2024-09-17T17:31:56.834621+0000","last_fullsized":"2024-09-17T17:31:56.834621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:22.541119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:22.541119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:24.275934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":272,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.221669+0000","last_change":"2024-09-17T17:31:50.178015+0000","last_active":"2024-09-17T17:31:56.221669+0000","last_peered":"2024-09-17T17:31:56.221669+0000","last_clean":"2024-09-17T17:31:56.221669+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:31:56.221669+0000","last_undegraded":"2024-09-17T17:31:56.221669+0000","last_fullsized":"2024-09-17T17:31:56.221669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:50.177917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:50.177917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:50.893244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178596+0000","last_change":"2024-09-17T17:31:56.070122+0000","last_active":"2024-09-17T17:31:56.178596+0000","last_peered":"2024-09-17T17:31:56.178596+0000","last_clean":"2024-09-17T17:31:56.178596+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:31:56.178596+0000","last_undegraded":"2024-09-17T17:31:56.178596+0000","last_fullsized":"2024-09-17T17:31:56.178596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-17T17:31:56.070038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:56.070038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.017596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183107+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:31:56.183107+0000","last_peered":"2024-09-17T17:31:56.183107+0000","last_clean":"2024-09-17T17:31:56.183107+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:31:56.183107+0000","last_undegraded":"2024-09-17T17:31:56.183107+0000","last_fullsized":"2024-09-17T17:31:56.183107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178594+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:31:56.178594+0000","last_peered":"2024-09-17T17:31:56.178594+0000","last_clean":"2024-09-17T17:31:56.178594+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:31:56.178594+0000","last_undegraded":"2024-09-17T17:31:56.178594+0000","last_fullsized":"2024-09-17T17:31:56.178594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183111+0000","last_change":"2024-09-17T17:31:49.471816+0000","last_active":"2024-09-17T17:31:56.183111+0000","last_peered":"2024-09-17T17:31:56.183111+0000","last_clean":"2024-09-17T17:31:56.183111+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:31:56.183111+0000","last_undegraded":"2024-09-17T17:31:56.183111+0000","last_fullsized":"2024-09-17T17:31:56.183111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:31:49.471731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.471731+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:54.889153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":404,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.254177+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:31:56.254177+0000","last_peered":"2024-09-17T17:31:56.254177+0000","last_clean":"2024-09-17T17:31:56.254177+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:31:56.254177+0000","last_undegraded":"2024-09-17T17:31:56.254177+0000","last_fullsized":"2024-09-17T17:31:56.254177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30820,"kb_used_data":2248,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94341020,"statfs":{"total":96636764160,"available":96605204480,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33968,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337872,"statfs":{"total":96636764160,"available":96601980928,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37604,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334236,"statfs":{"total":96636764160,"available":96598257664,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37608,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334232,"statfs":{"total":96636764160,"available":96598253568,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280547,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247842,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38116,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:32:00.423 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T17:32:00.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:32:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:32:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:32:00.736 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-17T17:32:00.188281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":612584,"kb_used_data":37016,"kb_used_omap":33,"kb_used_meta":575134,"kb_avail":1509336856,"statfs":{"total":1546188226560,"available":1545560940544,"internally_reserved":0,"allocated":37904384,"data_stored":22722822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":588938099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.025741"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182777+0000","last_change":"2024-09-17T17:31:49.863875+0000","last_active":"2024-09-17T17:31:56.182777+0000","last_peered":"2024-09-17T17:31:56.182777+0000","last_clean":"2024-09-17T17:31:56.182777+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:31:56.182777+0000","last_undegraded":"2024-09-17T17:31:56.182777+0000","last_fullsized":"2024-09-17T17:31:56.182777+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:49.863800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.863800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:53.247141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":269,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.182878+0000","last_change":"2024-09-17T17:31:34.252433+0000","last_active":"2024-09-17T17:31:56.182878+0000","last_peered":"2024-09-17T17:31:56.182878+0000","last_clean":"2024-09-17T17:31:56.182878+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:31:56.182878+0000","last_undegraded":"2024-09-17T17:31:56.182878+0000","last_fullsized":"2024-09-17T17:31:56.182878+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:34.252354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:34.252354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:01.572165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.834621+0000","last_change":"2024-09-17T17:31:22.541193+0000","last_active":"2024-09-17T17:31:56.834621+0000","last_peered":"2024-09-17T17:31:56.834621+0000","last_clean":"2024-09-17T17:31:56.834621+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:31:56.834621+0000","last_undegraded":"2024-09-17T17:31:56.834621+0000","last_fullsized":"2024-09-17T17:31:56.834621+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:22.541119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:22.541119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:24.275934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":272,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.221669+0000","last_change":"2024-09-17T17:31:50.178015+0000","last_active":"2024-09-17T17:31:56.221669+0000","last_peered":"2024-09-17T17:31:56.221669+0000","last_clean":"2024-09-17T17:31:56.221669+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:31:56.221669+0000","last_undegraded":"2024-09-17T17:31:56.221669+0000","last_fullsized":"2024-09-17T17:31:56.221669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:50.177917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:50.177917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:50.893244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178596+0000","last_change":"2024-09-17T17:31:56.070122+0000","last_active":"2024-09-17T17:31:56.178596+0000","last_peered":"2024-09-17T17:31:56.178596+0000","last_clean":"2024-09-17T17:31:56.178596+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:31:56.178596+0000","last_undegraded":"2024-09-17T17:31:56.178596+0000","last_fullsized":"2024-09-17T17:31:56.178596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-17T17:31:56.070038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:56.070038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.017596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":270,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183107+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:31:56.183107+0000","last_peered":"2024-09-17T17:31:56.183107+0000","last_clean":"2024-09-17T17:31:56.183107+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:31:56.183107+0000","last_undegraded":"2024-09-17T17:31:56.183107+0000","last_fullsized":"2024-09-17T17:31:56.183107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.178594+0000","last_change":"2024-09-17T17:30:47.793225+0000","last_active":"2024-09-17T17:31:56.178594+0000","last_peered":"2024-09-17T17:31:56.178594+0000","last_clean":"2024-09-17T17:31:56.178594+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:31:56.178594+0000","last_undegraded":"2024-09-17T17:31:56.178594+0000","last_fullsized":"2024-09-17T17:31:56.178594+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:47.793164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:47.793164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.626141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.183111+0000","last_change":"2024-09-17T17:31:49.471816+0000","last_active":"2024-09-17T17:31:56.183111+0000","last_peered":"2024-09-17T17:31:56.183111+0000","last_clean":"2024-09-17T17:31:56.183111+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:31:56.183111+0000","last_undegraded":"2024-09-17T17:31:56.183111+0000","last_fullsized":"2024-09-17T17:31:56.183111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:31:49.471731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.471731+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:54.889153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":404,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T17:31:56.254177+0000","last_change":"2024-09-17T17:30:38.330278+0000","last_active":"2024-09-17T17:31:56.254177+0000","last_peered":"2024-09-17T17:31:56.254177+0000","last_clean":"2024-09-17T17:31:56.254177+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:31:56.254177+0000","last_undegraded":"2024-09-17T17:31:56.254177+0000","last_fullsized":"2024-09-17T17:31:56.254177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:30:38.330144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:30:38.330144+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:01.582699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30820,"kb_used_data":2248,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94341020,"statfs":{"total":96636764160,"available":96605204480,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33968,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337872,"statfs":{"total":96636764160,"available":96601980928,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37604,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334236,"statfs":{"total":96636764160,"available":96598257664,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37608,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334232,"statfs":{"total":96636764160,"available":96598253568,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33440,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280547,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33440,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338400,"statfs":{"total":96636764160,"available":96602521600,"internally_reserved":0,"allocated":2306048,"data_stored":1354077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55652,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2301952,"data_stored":1354058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247842,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38116,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333724,"statfs":{"total":96636764160,"available":96597733376,"internally_reserved":0,"allocated":2834432,"data_stored":1882986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:32:00.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T17:32:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:32:01.046 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"fsid":"2593b77f-a53d-4649-b5a5-0122bd31e2b3","created":"2024-09-17T17:29:15.547925+0000","modified":"2024-09-17T17:32:00.183787+0000","last_up_change":"2024-09-17T17:31:19.959931+0000","last_in_change":"2024-09-17T17:30:22.332815+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T17:29:27.893486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T17:29:30.150043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72ee94e9-57bf-46b3-9fb9-0d05515fe1e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6817","nonce":50344815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6819","nonce":50344815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6823","nonce":50344815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":50344815},{"type":"v1","addr":"172.21.15.7:6821","nonce":50344815}]},"public_addr":"172.21.15.7:6817/50344815","cluster_addr":"172.21.15.7:6819/50344815","heartbeat_back_addr":"172.21.15.7:6823/50344815","heartbeat_front_addr":"172.21.15.7:6821/50344815","state":["exists","up"]},{"osd":1,"uuid":"e9764092-a458-4b6d-8bee-658f54f000cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6817","nonce":2426784045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6819","nonce":2426784045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6823","nonce":2426784045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2426784045},{"type":"v1","addr":"172.21.15.26:6821","nonce":2426784045}]},"public_addr":"172.21.15.26:6817/2426784045","cluster_addr":"172.21.15.26:6819/2426784045","heartbeat_back_addr":"172.21.15.26:6823/2426784045","heartbeat_front_addr":"172.21.15.26:6821/2426784045","state":["exists","up"]},{"osd":2,"uuid":"f1142f7d-9945-4daa-8f6a-6fd2d3090b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6801","nonce":1502510538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6803","nonce":1502510538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6807","nonce":1502510538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1502510538},{"type":"v1","addr":"172.21.15.27:6805","nonce":1502510538}]},"public_addr":"172.21.15.27:6801/1502510538","cluster_addr":"172.21.15.27:6803/1502510538","heartbeat_back_addr":"172.21.15.27:6807/1502510538","heartbeat_front_addr":"172.21.15.27:6805/1502510538","state":["exists","up"]},{"osd":3,"uuid":"daa4b149-c71a-4cba-a710-89aa101d7d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6811","nonce":2453627254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6813","nonce":2453627254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6817","nonce":2453627254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2453627254},{"type":"v1","addr":"172.21.15.102:6815","nonce":2453627254}]},"public_addr":"172.21.15.102:6811/2453627254","cluster_addr":"172.21.15.102:6813/2453627254","heartbeat_back_addr":"172.21.15.102:6817/2453627254","heartbeat_front_addr":"172.21.15.102:6815/2453627254","state":["exists","up"]},{"osd":4,"uuid":"55e8e034-493b-4066-a42b-f4d82294f3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6801","nonce":3507123258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6803","nonce":3507123258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6807","nonce":3507123258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3507123258},{"type":"v1","addr":"172.21.15.7:6805","nonce":3507123258}]},"public_addr":"172.21.15.7:6801/3507123258","cluster_addr":"172.21.15.7:6803/3507123258","heartbeat_back_addr":"172.21.15.7:6807/3507123258","heartbeat_front_addr":"172.21.15.7:6805/3507123258","state":["exists","up"]},{"osd":5,"uuid":"38a09763-d370-447a-b939-bad8a11c5e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6825","nonce":229014119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6827","nonce":229014119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6831","nonce":229014119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":229014119},{"type":"v1","addr":"172.21.15.26:6829","nonce":229014119}]},"public_addr":"172.21.15.26:6825/229014119","cluster_addr":"172.21.15.26:6827/229014119","heartbeat_back_addr":"172.21.15.26:6831/229014119","heartbeat_front_addr":"172.21.15.26:6829/229014119","state":["exists","up"]},{"osd":6,"uuid":"0ae069f9-8831-4bfc-9928-2100b73a839d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6817","nonce":3536708597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6819","nonce":3536708597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6823","nonce":3536708597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3536708597},{"type":"v1","addr":"172.21.15.27:6821","nonce":3536708597}]},"public_addr":"172.21.15.27:6817/3536708597","cluster_addr":"172.21.15.27:6819/3536708597","heartbeat_back_addr":"172.21.15.27:6823/3536708597","heartbeat_front_addr":"172.21.15.27:6821/3536708597","state":["exists","up"]},{"osd":7,"uuid":"a13c6ee0-8153-4ff4-b5e3-a6004a93ca0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6803","nonce":2367087286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6805","nonce":2367087286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6809","nonce":2367087286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2367087286},{"type":"v1","addr":"172.21.15.102:6807","nonce":2367087286}]},"public_addr":"172.21.15.102:6803/2367087286","cluster_addr":"172.21.15.102:6805/2367087286","heartbeat_back_addr":"172.21.15.102:6809/2367087286","heartbeat_front_addr":"172.21.15.102:6807/2367087286","state":["exists","up"]},{"osd":8,"uuid":"3c4daae0-6d93-450f-8600-b29362bd9f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":23,"last_clean_end":110,"up_from":117,"up_thru":117,"down_at":112,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6809","nonce":3844390318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6811","nonce":3844390318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6815","nonce":3844390318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3844390318},{"type":"v1","addr":"172.21.15.7:6813","nonce":3844390318}]},"public_addr":"172.21.15.7:6809/3844390318","cluster_addr":"172.21.15.7:6811/3844390318","heartbeat_back_addr":"172.21.15.7:6815/3844390318","heartbeat_front_addr":"172.21.15.7:6813/3844390318","state":["exists","up"]},{"osd":9,"uuid":"2846b3c6-06d6-414f-b4c9-b16d94007737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6809","nonce":2309011334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6811","nonce":2309011334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6815","nonce":2309011334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2309011334},{"type":"v1","addr":"172.21.15.26:6813","nonce":2309011334}]},"public_addr":"172.21.15.26:6809/2309011334","cluster_addr":"172.21.15.26:6811/2309011334","heartbeat_back_addr":"172.21.15.26:6815/2309011334","heartbeat_front_addr":"172.21.15.26:6813/2309011334","state":["exists","up"]},{"osd":10,"uuid":"495d7d7c-c222-4a46-9685-2a2a7cc6fc51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6809","nonce":2168900921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6811","nonce":2168900921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6815","nonce":2168900921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2168900921},{"type":"v1","addr":"172.21.15.27:6813","nonce":2168900921}]},"public_addr":"172.21.15.27:6809/2168900921","cluster_addr":"172.21.15.27:6811/2168900921","heartbeat_back_addr":"172.21.15.27:6815/2168900921","heartbeat_front_addr":"172.21.15.27:6813/2168900921","state":["exists","up"]},{"osd":11,"uuid":"cf76d383-65cd-48c3-a54c-cd83ffbc9898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6827","nonce":3109125505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6829","nonce":3109125505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6833","nonce":3109125505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3109125505},{"type":"v1","addr":"172.21.15.102:6831","nonce":3109125505}]},"public_addr":"172.21.15.102:6827/3109125505","cluster_addr":"172.21.15.102:6829/3109125505","heartbeat_back_addr":"172.21.15.102:6833/3109125505","heartbeat_front_addr":"172.21.15.102:6831/3109125505","state":["exists","up"]},{"osd":12,"uuid":"54294878-ea1e-4845-a0e6-9be026f521fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6825","nonce":3659919075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6827","nonce":3659919075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6831","nonce":3659919075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3659919075},{"type":"v1","addr":"172.21.15.7:6829","nonce":3659919075}]},"public_addr":"172.21.15.7:6825/3659919075","cluster_addr":"172.21.15.7:6827/3659919075","heartbeat_back_addr":"172.21.15.7:6831/3659919075","heartbeat_front_addr":"172.21.15.7:6829/3659919075","state":["exists","up"]},{"osd":13,"uuid":"9acf83a5-75db-4874-8f3f-7421cc55e2fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6801","nonce":1983765258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6803","nonce":1983765258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6807","nonce":1983765258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1983765258},{"type":"v1","addr":"172.21.15.26:6805","nonce":1983765258}]},"public_addr":"172.21.15.26:6801/1983765258","cluster_addr":"172.21.15.26:6803/1983765258","heartbeat_back_addr":"172.21.15.26:6807/1983765258","heartbeat_front_addr":"172.21.15.26:6805/1983765258","state":["exists","up"]},{"osd":14,"uuid":"16c02ebe-a418-4f87-902d-b5dac864f139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6825","nonce":1199123829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6827","nonce":1199123829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6831","nonce":1199123829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1199123829},{"type":"v1","addr":"172.21.15.27:6829","nonce":1199123829}]},"public_addr":"172.21.15.27:6825/1199123829","cluster_addr":"172.21.15.27:6827/1199123829","heartbeat_back_addr":"172.21.15.27:6831/1199123829","heartbeat_front_addr":"172.21.15.27:6829/1199123829","state":["exists","up"]},{"osd":15,"uuid":"a61ebbd1-f5bd-4fd8-806a-3cc7592ebfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6819","nonce":259571866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6821","nonce":259571866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6825","nonce":259571866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":259571866},{"type":"v1","addr":"172.21.15.102:6823","nonce":259571866}]},"public_addr":"172.21.15.102:6819/259571866","cluster_addr":"172.21.15.102:6821/259571866","heartbeat_back_addr":"172.21.15.102:6825/259571866","heartbeat_front_addr":"172.21.15.102:6823/259571866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:53.149089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:29.488817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:53.727934+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:59.891154+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:59.913787+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:46.730260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:48.564572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:46.687618+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:46.536431+0000","dead_epoch":112},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:44.042967+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:33.448841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:41.027274+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:33.749926+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.511757+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:30:48.711104+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T17:31:48.758623+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T17:32:02.060 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T17:32:02.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:02.212 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:02.212 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:02.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T17:32:02.530 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T17:32:02.544 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T17:32:02.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:02.681 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:02.681 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:02.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T17:32:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T17:32:03.012 INFO:tasks.ceph:Scrubbing osd.8 2024-09-17T17:32:03.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:03.149 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:03.150 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-17T17:32:03.150 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:03.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-17T17:32:03.465 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 8 to deep-scrub 2024-09-17T17:32:03.479 INFO:tasks.ceph:Scrubbing osd.12 2024-09-17T17:32:03.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:03.616 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:03.616 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:03.616 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:03.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-09-17T17:32:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 12 to deep-scrub 2024-09-17T17:32:03.944 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T17:32:03.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:04.082 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:04.083 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:04.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T17:32:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T17:32:04.412 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T17:32:04.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:04.549 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:04.550 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:04.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T17:32:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T17:32:04.883 INFO:tasks.ceph:Scrubbing osd.9 2024-09-17T17:32:04.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:05.020 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:05.020 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:05.020 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:05.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-17T17:32:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 9 to deep-scrub 2024-09-17T17:32:05.349 INFO:tasks.ceph:Scrubbing osd.13 2024-09-17T17:32:05.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:05.487 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:05.488 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:05.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-09-17T17:32:05.802 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 13 to deep-scrub 2024-09-17T17:32:05.815 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T17:32:05.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:05.952 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:05.953 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:05.953 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:05.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T17:32:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T17:32:06.284 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T17:32:06.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:06.421 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:06.422 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:06.422 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:06.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T17:32:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T17:32:06.751 INFO:tasks.ceph:Scrubbing osd.10 2024-09-17T17:32:06.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:06.889 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:06.889 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:06.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-17T17:32:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 10 to deep-scrub 2024-09-17T17:32:07.215 INFO:tasks.ceph:Scrubbing osd.14 2024-09-17T17:32:07.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:07.353 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:07.353 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:07.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-09-17T17:32:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 14 to deep-scrub 2024-09-17T17:32:07.683 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T17:32:07.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:07.821 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:07.821 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:07.821 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:07.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T17:32:08.134 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T17:32:08.147 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T17:32:08.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:08.284 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:08.285 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:08.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T17:32:08.598 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T17:32:08.613 INFO:tasks.ceph:Scrubbing osd.11 2024-09-17T17:32:08.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:08.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:08.753 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:08.753 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:08.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-17T17:32:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 11 to deep-scrub 2024-09-17T17:32:09.085 INFO:tasks.ceph:Scrubbing osd.15 2024-09-17T17:32:09.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-09-17T17:32:09.223 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T17:32:09.223 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T17:32:09.223 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T17:32:09.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-09-17T17:32:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 15 to deep-scrub 2024-09-17T17:32:09.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:32:09.860 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:32:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:32:09.873 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-17T17:32:07.902606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":617224,"kb_used_data":37560,"kb_used_omap":33,"kb_used_meta":579294,"kb_avail":1509332216,"statfs":{"total":1546188226560,"available":1545556189184,"internally_reserved":0,"allocated":38461440,"data_stored":23050974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":593197939},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.714325"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":272,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:02.531899+0000","last_change":"2024-09-17T17:31:49.863875+0000","last_active":"2024-09-17T17:32:02.531899+0000","last_peered":"2024-09-17T17:32:02.531899+0000","last_clean":"2024-09-17T17:32:02.531899+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:32:02.531899+0000","last_undegraded":"2024-09-17T17:32:02.531899+0000","last_fullsized":"2024-09-17T17:32:02.531899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:49.863800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.863800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:53.247141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":273,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:02.381730+0000","last_change":"2024-09-17T17:31:34.252433+0000","last_active":"2024-09-17T17:32:02.381730+0000","last_peered":"2024-09-17T17:32:02.381730+0000","last_clean":"2024-09-17T17:32:02.381730+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:32:02.381730+0000","last_undegraded":"2024-09-17T17:32:02.381730+0000","last_fullsized":"2024-09-17T17:32:02.381730+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:31:34.252354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:34.252354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:01.572165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:03.611252+0000","last_change":"2024-09-17T17:32:03.611252+0000","last_active":"2024-09-17T17:32:03.611252+0000","last_peered":"2024-09-17T17:32:03.611252+0000","last_clean":"2024-09-17T17:32:03.611252+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:32:03.611252+0000","last_undegraded":"2024-09-17T17:32:03.611252+0000","last_fullsized":"2024-09-17T17:32:03.611252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:03.611175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:03.611175+0000","last_clean_scrub_stamp":"2024-09-17T17:32:03.611175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:28.288823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T17:32:05.408416+0000","last_change":"2024-09-17T17:32:05.324921+0000","last_active":"2024-09-17T17:32:05.408416+0000","last_peered":"2024-09-17T17:32:05.408416+0000","last_clean":"2024-09-17T17:32:05.408416+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:32:05.408416+0000","last_undegraded":"2024-09-17T17:32:05.408416+0000","last_fullsized":"2024-09-17T17:32:05.408416+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:05.324830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:05.324830+0000","last_clean_scrub_stamp":"2024-09-17T17:32:05.324830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:14.369700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:01.383736+0000","last_change":"2024-09-17T17:31:56.070122+0000","last_active":"2024-09-17T17:32:01.383736+0000","last_peered":"2024-09-17T17:32:01.383736+0000","last_clean":"2024-09-17T17:32:01.383736+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:32:01.383736+0000","last_undegraded":"2024-09-17T17:32:01.383736+0000","last_fullsized":"2024-09-17T17:32:01.383736+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-17T17:31:56.070038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:56.070038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.017596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:02.383016+0000","last_change":"2024-09-17T17:30:51.558349+0000","last_active":"2024-09-17T17:32:02.383016+0000","last_peered":"2024-09-17T17:32:02.383016+0000","last_clean":"2024-09-17T17:32:02.383016+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:32:02.383016+0000","last_undegraded":"2024-09-17T17:32:02.383016+0000","last_fullsized":"2024-09-17T17:32:02.383016+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:30:51.558171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:30:51.558171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:12.578797+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:02.978915+0000","last_change":"2024-09-17T17:32:02.978915+0000","last_active":"2024-09-17T17:32:02.978915+0000","last_peered":"2024-09-17T17:32:02.978915+0000","last_clean":"2024-09-17T17:32:02.978915+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:32:02.978915+0000","last_undegraded":"2024-09-17T17:32:02.978915+0000","last_fullsized":"2024-09-17T17:32:02.978915+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:02.978822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:32:02.978822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:28.773856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":283,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:02.383009+0000","last_change":"2024-09-17T17:31:49.471816+0000","last_active":"2024-09-17T17:32:02.383009+0000","last_peered":"2024-09-17T17:32:02.383009+0000","last_clean":"2024-09-17T17:32:02.383009+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:32:02.383009+0000","last_undegraded":"2024-09-17T17:32:02.383009+0000","last_fullsized":"2024-09-17T17:32:02.383009+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-17T17:31:49.471731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:30.568245+0000","last_clean_scrub_stamp":"2024-09-17T17:31:49.471731+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:54.889153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"25'112","reported_seq":416,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T17:32:02.497813+0000","last_change":"2024-09-17T17:32:02.497813+0000","last_active":"2024-09-17T17:32:02.497813+0000","last_peered":"2024-09-17T17:32:02.497813+0000","last_clean":"2024-09-17T17:32:02.497813+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:32:02.497813+0000","last_undegraded":"2024-09-17T17:32:02.497813+0000","last_fullsized":"2024-09-17T17:32:02.497813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'112","last_scrub_stamp":"2024-09-17T17:32:02.497719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:29:28.558396+0000","last_clean_scrub_stamp":"2024-09-17T17:32:02.497719+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.753955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55680,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316160,"statfs":{"total":96636764160,"available":96579747840,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173643,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30844,"kb_used_data":2280,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94340996,"statfs":{"total":96636764160,"available":96605179904,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313252,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1382992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280547,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313253,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33484,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338356,"statfs":{"total":96636764160,"available":96602476544,"internally_reserved":0,"allocated":2351104,"data_stored":1382992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280548,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2867200,"data_stored":1902292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37632,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334208,"statfs":{"total":96636764160,"available":96598228992,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280548,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37636,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334204,"statfs":{"total":96636764160,"available":96598224896,"internally_reserved":0,"allocated":2338816,"data_stored":1373383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280547,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33468,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338372,"statfs":{"total":96636764160,"available":96602492928,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33472,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280548,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33476,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338364,"statfs":{"total":96636764160,"available":96602484736,"internally_reserved":0,"allocated":2338816,"data_stored":1373383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55680,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316160,"statfs":{"total":96636764160,"available":96579747840,"internally_reserved":0,"allocated":2334720,"data_stored":1373364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247843,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38144,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333696,"statfs":{"total":96636764160,"available":96597704704,"internally_reserved":0,"allocated":2867200,"data_stored":1902292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:32:09.877 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T17:31:49.863800+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=31, tm_sec=49, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=32, tm_sec=1, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T17:32:09.877 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T17:31:34.252354+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=31, tm_sec=34, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=32, tm_sec=1, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T17:32:09.877 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T17:31:56.070038+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=31, tm_sec=56, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=32, tm_sec=1, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T17:32:09.877 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T17:30:51.558171+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=30, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=32, tm_sec=1, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T17:32:09.877 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T17:31:49.471731+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=31, tm_sec=49, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=17, tm_min=32, tm_sec=1, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T17:32:09.878 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T17:32:29.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T17:32:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T17:32:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T17:32:30.209 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-17T17:32:29.907559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":618216,"kb_used_data":38552,"kb_used_omap":33,"kb_used_meta":579294,"kb_avail":1509331224,"statfs":{"total":1546188226560,"available":1545555173376,"internally_reserved":0,"allocated":39477248,"data_stored":23647910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33933,"internal_metadata":593197939},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.646249"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:19.262532+0000","last_change":"2024-09-17T17:32:09.844480+0000","last_active":"2024-09-17T17:32:19.262532+0000","last_peered":"2024-09-17T17:32:19.262532+0000","last_clean":"2024-09-17T17:32:19.262532+0000","last_became_active":"2024-09-17T17:29:32.420156+0000","last_became_peered":"2024-09-17T17:29:32.420156+0000","last_unstale":"2024-09-17T17:32:19.262532+0000","last_undegraded":"2024-09-17T17:32:19.262532+0000","last_fullsized":"2024-09-17T17:32:19.262532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:09.844396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:09.844396+0000","last_clean_scrub_stamp":"2024-09-17T17:32:09.844396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:30.207389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047762800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":289,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:19.637815+0000","last_change":"2024-09-17T17:32:06.163366+0000","last_active":"2024-09-17T17:32:19.637815+0000","last_peered":"2024-09-17T17:32:19.637815+0000","last_clean":"2024-09-17T17:32:19.637815+0000","last_became_active":"2024-09-17T17:29:31.584646+0000","last_became_peered":"2024-09-17T17:29:31.584646+0000","last_unstale":"2024-09-17T17:32:19.637815+0000","last_undegraded":"2024-09-17T17:32:19.637815+0000","last_fullsized":"2024-09-17T17:32:19.637815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:06.163287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:06.163287+0000","last_clean_scrub_stamp":"2024-09-17T17:32:06.163287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:14.011552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041326299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:20.887772+0000","last_change":"2024-09-17T17:32:03.611252+0000","last_active":"2024-09-17T17:32:20.887772+0000","last_peered":"2024-09-17T17:32:20.887772+0000","last_clean":"2024-09-17T17:32:20.887772+0000","last_became_active":"2024-09-17T17:31:21.304552+0000","last_became_peered":"2024-09-17T17:31:21.304552+0000","last_unstale":"2024-09-17T17:32:20.887772+0000","last_undegraded":"2024-09-17T17:32:20.887772+0000","last_fullsized":"2024-09-17T17:32:20.887772+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:03.611175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:03.611175+0000","last_clean_scrub_stamp":"2024-09-17T17:32:03.611175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:28.288823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":292,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:20.057723+0000","last_change":"2024-09-17T17:32:05.324921+0000","last_active":"2024-09-17T17:32:20.057723+0000","last_peered":"2024-09-17T17:32:20.057723+0000","last_clean":"2024-09-17T17:32:20.057723+0000","last_became_active":"2024-09-17T17:29:31.585287+0000","last_became_peered":"2024-09-17T17:29:31.585287+0000","last_unstale":"2024-09-17T17:32:20.057723+0000","last_undegraded":"2024-09-17T17:32:20.057723+0000","last_fullsized":"2024-09-17T17:32:20.057723+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:05.324830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:05.324830+0000","last_clean_scrub_stamp":"2024-09-17T17:32:05.324830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:14.369700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000440939,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"153'3","reported_seq":291,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:19.261746+0000","last_change":"2024-09-17T17:32:09.928515+0000","last_active":"2024-09-17T17:32:19.261746+0000","last_peered":"2024-09-17T17:32:19.261746+0000","last_clean":"2024-09-17T17:32:19.261746+0000","last_became_active":"2024-09-17T17:29:31.584911+0000","last_became_peered":"2024-09-17T17:29:31.584911+0000","last_unstale":"2024-09-17T17:32:19.261746+0000","last_undegraded":"2024-09-17T17:32:19.261746+0000","last_fullsized":"2024-09-17T17:32:19.261746+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'3","last_scrub_stamp":"2024-09-17T17:32:09.928436+0000","last_deep_scrub":"153'3","last_deep_scrub_stamp":"2024-09-17T17:32:09.928436+0000","last_clean_scrub_stamp":"2024-09-17T17:32:09.928436+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:32.969096+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00044003299999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:19.624533+0000","last_change":"2024-09-17T17:32:09.604800+0000","last_active":"2024-09-17T17:32:19.624533+0000","last_peered":"2024-09-17T17:32:19.624533+0000","last_clean":"2024-09-17T17:32:19.624533+0000","last_became_active":"2024-09-17T17:29:31.588751+0000","last_became_peered":"2024-09-17T17:29:31.588751+0000","last_unstale":"2024-09-17T17:32:19.624533+0000","last_undegraded":"2024-09-17T17:32:19.624533+0000","last_fullsized":"2024-09-17T17:32:19.624533+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:09.604678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:09.604678+0000","last_clean_scrub_stamp":"2024-09-17T17:32:09.604678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:24.775607+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00041492100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":296,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:19.261750+0000","last_change":"2024-09-17T17:32:08.923702+0000","last_active":"2024-09-17T17:32:19.261750+0000","last_peered":"2024-09-17T17:32:19.261750+0000","last_clean":"2024-09-17T17:32:19.261750+0000","last_became_active":"2024-09-17T17:29:31.583429+0000","last_became_peered":"2024-09-17T17:29:31.583429+0000","last_unstale":"2024-09-17T17:32:19.261750+0000","last_undegraded":"2024-09-17T17:32:19.261750+0000","last_fullsized":"2024-09-17T17:32:19.261750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T17:32:08.923614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T17:32:08.923614+0000","last_clean_scrub_stamp":"2024-09-17T17:32:08.923614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:25.699765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044680499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"153'2","reported_seq":300,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:19.624531+0000","last_change":"2024-09-17T17:32:08.562713+0000","last_active":"2024-09-17T17:32:19.624531+0000","last_peered":"2024-09-17T17:32:19.624531+0000","last_clean":"2024-09-17T17:32:19.624531+0000","last_became_active":"2024-09-17T17:29:31.588640+0000","last_became_peered":"2024-09-17T17:29:31.588640+0000","last_unstale":"2024-09-17T17:32:19.624531+0000","last_undegraded":"2024-09-17T17:32:19.624531+0000","last_fullsized":"2024-09-17T17:32:19.624531+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'2","last_scrub_stamp":"2024-09-17T17:32:08.562617+0000","last_deep_scrub":"153'2","last_deep_scrub_stamp":"2024-09-17T17:32:08.562617+0000","last_clean_scrub_stamp":"2024-09-17T17:32:08.562617+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:22.036009+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00040547800000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"153'114","reported_seq":434,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T17:32:20.057881+0000","last_change":"2024-09-17T17:32:07.608669+0000","last_active":"2024-09-17T17:32:20.057881+0000","last_peered":"2024-09-17T17:32:20.057881+0000","last_clean":"2024-09-17T17:32:20.057881+0000","last_became_active":"2024-09-17T17:29:29.847008+0000","last_became_peered":"2024-09-17T17:29:29.847008+0000","last_unstale":"2024-09-17T17:32:20.057881+0000","last_undegraded":"2024-09-17T17:32:20.057881+0000","last_fullsized":"2024-09-17T17:32:20.057881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'114","last_scrub_stamp":"2024-09-17T17:32:07.608580+0000","last_deep_scrub":"153'114","last_deep_scrub_stamp":"2024-09-17T17:32:07.608580+0000","last_clean_scrub_stamp":"2024-09-17T17:32:07.608580+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:11.468859+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":12,"up_from":23,"seq":98784247848,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55740,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316100,"statfs":{"total":96636764160,"available":96579686400,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":23,"seq":98784247848,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33532,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":117,"seq":502511173648,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30908,"kb_used_data":2344,"kb_used_omap":9,"kb_used_meta":28534,"kb_avail":94340932,"statfs":{"total":96636764160,"available":96605114368,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10007,"internal_metadata":29219049},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313257,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59904,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311936,"statfs":{"total":96636764160,"available":96575422464,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280552,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33532,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":21,"seq":90194313257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33532,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,5,6,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34056,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337784,"statfs":{"total":96636764160,"available":96601890816,"internally_reserved":0,"allocated":2932736,"data_stored":1940804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33532,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280553,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[3,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37696,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334144,"statfs":{"total":96636764160,"available":96598163456,"internally_reserved":0,"allocated":2404352,"data_stored":1411895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,2,3,4,8,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33532,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,3,4,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280552,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33536,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338304,"statfs":{"total":96636764160,"available":96602423296,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33532,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33536,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338304,"statfs":{"total":96636764160,"available":96602423296,"internally_reserved":0,"allocated":2404352,"data_stored":1411895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247849,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55740,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316100,"statfs":{"total":96636764160,"available":96579686400,"internally_reserved":0,"allocated":2400256,"data_stored":1411876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[2,3,5,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247848,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38216,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333624,"statfs":{"total":96636764160,"available":96597630976,"internally_reserved":0,"allocated":2932736,"data_stored":1940804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,7,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T17:32:30.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T17:32:30.535 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T17:32:30.535 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T17:32:30.536 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T17:32:30.536 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.581 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T17:32:30.581 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T17:32:30.582 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.619 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T17:32:30.619 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-17T17:32:30.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.695 INFO:tasks.ceph.osd.8:Stopped 2024-09-17T17:32:30.695 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-09-17T17:32:30.695 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.767 INFO:tasks.ceph.osd.12:Stopped 2024-09-17T17:32:30.768 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T17:32:30.768 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.808 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T17:32:30.808 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T17:32:30.808 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.881 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T17:32:30.881 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-17T17:32:30.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:30.961 INFO:tasks.ceph.osd.9:Stopped 2024-09-17T17:32:30.961 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-09-17T17:32:30.962 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.031 INFO:tasks.ceph.osd.13:Stopped 2024-09-17T17:32:31.032 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T17:32:31.032 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.065 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T17:32:31.066 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T17:32:31.066 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.139 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T17:32:31.140 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-17T17:32:31.140 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.215 INFO:tasks.ceph.osd.10:Stopped 2024-09-17T17:32:31.216 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-09-17T17:32:31.216 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.285 INFO:tasks.ceph.osd.14:Stopped 2024-09-17T17:32:31.285 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T17:32:31.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.322 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T17:32:31.322 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T17:32:31.322 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.398 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T17:32:31.399 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-17T17:32:31.399 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.475 INFO:tasks.ceph.osd.11:Stopped 2024-09-17T17:32:31.476 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-09-17T17:32:31.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.543 INFO:tasks.ceph.osd.15:Stopped 2024-09-17T17:32:31.544 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T17:32:31.544 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T17:32:31.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.571 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T17:32:31.571 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T17:32:31.571 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.617 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T17:32:31.617 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T17:32:31.617 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T17:32:31.617 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.629 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T17:32:31.630 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T17:32:31.630 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.644 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T17:32:31.644 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T17:32:31.644 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T17:32:31.658 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T17:32:31.659 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T17:32:31.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T17:32:31.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:32:31.708 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T17:32:31.823 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:32:31.823 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T17:32:31.922 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:32:31.923 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-17T17:32:31.971 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:32:31.971 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-09-17T17:32:32.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:32:32.056 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T17:32:32.175 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:32:32.175 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T17:32:32.257 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:32:32.257 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-17T17:32:32.314 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:32:32.314 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-09-17T17:32:32.371 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:32:32.371 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T17:32:32.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:32:32.478 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T17:32:32.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:32:32.567 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-17T17:32:32.615 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:32:32.615 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-09-17T17:32:32.673 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:32:32.674 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T17:32:32.741 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:32:32.741 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T17:32:32.821 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:32:32.821 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-17T17:32:32.885 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:32:32.885 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-09-17T17:32:32.946 INFO:tasks.ceph:Archiving mon data... 2024-09-17T17:32:32.947 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/data/mon.a.tgz 2024-09-17T17:32:32.948 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-17T17:32:32.967 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.EjPmx71Tju 2024-09-17T17:32:32.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.EjPmx71Tju 2024-09-17T17:32:33.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.EjPmx71Tju 2024-09-17T17:32:33.235 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.EjPmx71Tju is 471KB 2024-09-17T17:32:33.334 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.EjPmx71Tju 2024-09-17T17:32:33.349 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/data/mon.b.tgz 2024-09-17T17:32:33.350 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-17T17:32:33.369 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.dr8943qdYt 2024-09-17T17:32:33.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.dr8943qdYt 2024-09-17T17:32:33.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.dr8943qdYt 2024-09-17T17:32:33.681 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.dr8943qdYt is 675KB 2024-09-17T17:32:33.781 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.dr8943qdYt 2024-09-17T17:32:33.797 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/data/mon.c.tgz 2024-09-17T17:32:33.798 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-17T17:32:33.817 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.4fLVNpt9Zt 2024-09-17T17:32:33.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4fLVNpt9Zt 2024-09-17T17:32:34.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.4fLVNpt9Zt 2024-09-17T17:32:34.123 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.4fLVNpt9Zt is 675KB 2024-09-17T17:32:34.225 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.4fLVNpt9Zt 2024-09-17T17:32:34.241 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T17:32:34.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T17:32:34.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T17:32:34.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T17:32:34.283 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T17:32:34.333 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T17:32:34.351 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T17:32:34.366 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T17:32:34.384 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T17:32:34.385 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T17:32:34.386 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi007/crash 2024-09-17T17:32:34.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T17:32:34.417 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi026/crash 2024-09-17T17:32:34.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T17:32:34.447 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi027/crash 2024-09-17T17:32:34.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T17:32:34.473 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi102/crash 2024-09-17T17:32:34.474 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T17:32:34.529 INFO:tasks.ceph:Compressing logs... 2024-09-17T17:32:34.529 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T17:32:34.532 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T17:32:34.535 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T17:32:34.537 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T17:32:34.565 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-09-17T17:32:34.565 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T17:32:34.565 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T17:32:34.565 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.6.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.log: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T17:32:34.566 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T17:32:34.567 INFO:teuthology.orchestra.run.smithi026.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32671.log 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32918.log 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33158.log 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32671.log.gz 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32918.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32918.log.gz 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32838.log 2024-09-17T17:32:34.580 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33158.log.gz 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32958.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32838.log.gz 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32627.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32798.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32958.log.gz 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32627.log.gz 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33198.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32758.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32798.log.gz 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33198.log.gz 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32718.log 2024-09-17T17:32:34.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32758.log.gz 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32718.log.gz 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33118.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32998.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33118.log.gz 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32878.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33038.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32998.log.gz 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32878.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32878.log.gz 2024-09-17T17:32:34.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38019.log 2024-09-17T17:32:34.583 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33038.log.gz 2024-09-17T17:32:34.583 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32587.log 2024-09-17T17:32:34.583 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.0.38019.log: 77.7% -- replaced with /var/log/ceph/ceph-client.0.38019.log.gz 2024-09-17T17:32:34.583 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32587.log.gz 2024-09-17T17:32:34.627 INFO:teuthology.orchestra.run.smithi102.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T17:32:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T17:32:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48841.log 2024-09-17T17:32:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53461.log 2024-09-17T17:32:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/objectstore_tool.48841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54075.log 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53461.log.gz 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41650.log 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54075.log.gz 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38346.log 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41650.log.gz 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48242.log 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-09-17T17:32:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38346.log.gz 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39449.log 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37781.log 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48242.log.gz 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39449.log.gz 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38857.log 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38324.log 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37781.log.gz 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38857.log.gz 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53100.log 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43835.log 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38324.log.gz 2024-09-17T17:32:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53100.log: 27.9% -- replaced with /var/log/ceph/ceph-client.admin.53100.log.gz 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55116.log 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43835.log.gz 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55116.log.gz 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55494.log 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39033.log 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55494.log.gz 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39033.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39033.log.gz -5 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33417.log 2024-09-17T17:32:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51756.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33417.log.gz 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51756.log.gz 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41692.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37981.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41692.log.gz 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42327.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40981.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37981.log.gz 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-09-17T17:32:34.693 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42327.log.gz 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40981.log.gz 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54129.log 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53458.log 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54129.log.gz 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55872.log 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41096.log 2024-09-17T17:32:34.694 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53458.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55872.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39577.log 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41096.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39577.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54864.log 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46343.log 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52092.log 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54864.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46343.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46343.log.gz 2024-09-17T17:32:34.695 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46401.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38329.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38352.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46401.log.gz 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38329.log.gz 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48690.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38352.log: 94.4% -- replaced with /var/log/ceph/objectstore_tool.48841.log.gz 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55452.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38352.log.gz 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53469.log 2024-09-17T17:32:34.696 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48690.log.gz 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55452.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38343.log 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55452.log.gz 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53455.log 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53469.log.gz 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38343.log.gz 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43480.log 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53455.log.gz 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46253.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz --verbose 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.47781.log 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42174.log 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43480.log.gz 2024-09-17T17:32:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55032.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47781.log.gz 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46213.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42174.log.gz 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55032.log.gz 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46213.log.gz 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44413.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54990.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44413.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2024-09-17T17:32:34.698 INFO:teuthology.orchestra.run.smithi007.stderr: 24.4% -- replaced with /var/log/ceph/ceph-client.admin.44413.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45449.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54990.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53951.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53058.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45449.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53951.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53058.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56206.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43786.log: 24.0% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-09-17T17:32:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42415.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56206.log.gz 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47028.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52680.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42415.log.gz 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47028.log: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.47028.log.gz 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54822.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38023.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52680.log.gz 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54822.log.gz 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45235.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2024-09-17T17:32:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38023.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38894.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45235.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38345.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38894.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47604.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38345.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54612.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47604.log.gz 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55158.log 2024-09-17T17:32:34.701 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54612.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39409.log 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52302.log 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55158.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39409.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39409.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54528.log 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41179.log 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52302.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38338.log 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54528.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46776.log 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr: 24.4% -- replaced with /var/log/ceph/ceph-client.admin.41179.log.gz 2024-09-17T17:32:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38338.log.gz 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55746.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.46776.log.gz 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40511.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40228.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55746.log.gz 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38342.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54144.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40228.log.gz 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38342.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38342.log.gz -5 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43136.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-09-17T17:32:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54144.log.gz 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52764.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43136.log.gz 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44528.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52764.log.gz 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44528.log: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.44528.log.gz 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54133.log 2024-09-17T17:32:34.704 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54133.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38782.log 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54133.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54738.log 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45491.log: 25.0% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38782.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41138.log 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54738.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53451.log 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41138.log.gz 2024-09-17T17:32:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47688.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53451.log.gz 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr: 42.7% -- replaced with /var/log/ceph/ceph-client.admin.47688.log.gz 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55998.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40399.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55998.log.gz 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37725.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40399.log.gz 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43966.log 2024-09-17T17:32:34.706 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37725.log.gz 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41787.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gzgzip 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56082.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41787.log.gz 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56082.log.gz 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39703.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39619.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44998.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54053.log 2024-09-17T17:32:34.707 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39703.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53473.log 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39619.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48472.log 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54053.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43033.log 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53473.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48472.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40352.log 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43033.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39970.log 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-09-17T17:32:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33537.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40352.log.gz 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40626.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.39970.log.gz 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33537.log.gz 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40626.log.gz 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38970.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39489.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz -5 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55662.log 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38970.log.gz 2024-09-17T17:32:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48283.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz -5 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56040.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47161.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55662.log.gz 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56040.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.56040.log.gz -- 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.53480.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54101.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47161.log.gz 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53480.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.53480.log.gz --verbose 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.42216.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43179.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54101.log.gz 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42216.log.gz 2024-09-17T17:32:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43179.log.gz 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51798.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz -5 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/objectstore_tool.48526.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41023.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.48526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42776.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53905.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41023.log.gz 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56166.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42776.log.gz 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53905.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53905.log.gz 2024-09-17T17:32:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52722.log 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56166.log.gz 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52722.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52722.log.gz 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39008.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.50017.log 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2024-09-17T17:32:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.50017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55914.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55914.log.gz 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38327.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38964.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38327.log.gz 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52512.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38964.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54004.log 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38964.log.gz 2024-09-17T17:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47562.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52512.log: /var/log/ceph/ceph-client.admin.54004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42979.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52512.log.gz 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54004.log.gz 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47562.log.gz 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52596.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42979.log: -- /var/log/ceph/ceph-client.admin.54127.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.42979.log.gz/var/log/ceph/ceph-client.admin.52596.log: 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52596.log.gz 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52848.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53484.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54127.log: /var/log/ceph/ceph-client.admin.52848.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52848.log.gz 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.38309.log 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0%/var/log/ceph/ceph-client.admin.53484.log: -- replaced with /var/log/ceph/ceph-client.admin.54127.log.gz 2024-09-17T17:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53448.log 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38309.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39009.log 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53448.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53448.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55074.log 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39009.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39009.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55074.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39019.log 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41712.log: 26.6% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-09-17T17:32:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46514.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39019.log: /var/log/ceph/ceph-client.admin.52008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39745.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39019.log.gz 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46514.log.gz 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55242.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55830.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39745.log: /var/log/ceph/ceph-client.admin.55242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53466.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39745.log.gz 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55242.log.gz 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55830.log.gz 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53460.log 2024-09-17T17:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53466.log: /var/log/ceph/ceph-client.admin.53460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53466.log.gz 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27304.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53460.log.gz 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49385.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45193.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.27304.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27304.log.gz 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49385.log.gz 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38986.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55704.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45193.log.gz 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38986.log.gz 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40888.log 2024-09-17T17:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55704.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52806.log 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54047.log 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40888.log: 24.8% -- replaced with /var/log/ceph/ceph-client.admin.40888.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52806.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52344.log 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr: 94.4% -- replaced with /var/log/ceph/objectstore_tool.48526.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38989.log 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54047.log: /var/log/ceph/ceph-client.admin.52344.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43301.log 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52344.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54047.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38989.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38989.log.gz 2024-09-17T17:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53481.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43301.log: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.43301.log.gz 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37818.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53481.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53481.log.gz 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47987.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37818.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39043.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46293.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.47987.log.gz 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53474.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr: 93.6% -- replaced with /var/log/ceph/objectstore_tool.50017.log.gz 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39043.log: /var/log/ceph/ceph-client.admin.46293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39043.log.gz 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-09-17T17:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.46293.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44956.log 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53474.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44956.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 0.0% 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45141.log 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr: 24.0% -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2024-09-17T17:32:34.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38356.log 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43644.log 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45141.log.gz 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53142.log 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38356.log.gz 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43644.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-mgr.y.log -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53016.log 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51924.log 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42497.log 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53016.log.gz 2024-09-17T17:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51924.log.gz 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42497.log: -- /var/log/ceph/ceph-client.admin.55368.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.42497.log.gz 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42236.log: gzip 24.3% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43397.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55368.log.gz 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47008.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33150.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43397.log.gz 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47008.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47008.log.gz 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33150.log: -- /var/log/ceph/ceph-client.admin.46893.log 2024-09-17T17:32:34.722 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33150.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52974.log 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53906.log 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr: 25.1% -- replaced with /var/log/ceph/ceph-client.admin.46893.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46944.log 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52974.log: /var/log/ceph/ceph-client.admin.53906.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52974.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53906.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47927.log 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46944.log: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46944.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45802.log 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47927.log.gz 2024-09-17T17:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52386.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54570.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr: 25.2% -- replaced with /var/log/ceph/ceph-client.admin.45802.log.gz 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52386.log.gz 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54570.log.gz 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39849.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55620.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.49134.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54258.log 2024-09-17T17:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55620.log.gz 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.49134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38361.log 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54258.log.gz 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39007.log 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44601.log 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38361.log: /var/log/ceph/ceph-client.admin.39007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55326.log 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38361.log.gz 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39007.log.gz 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44601.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39661.log 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44601.log.gz 2024-09-17T17:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55326.log.gz 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52470.log 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39661.log: gzip/var/log/ceph/ceph-client.admin.44486.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.53425.log 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0%/var/log/ceph/ceph-client.admin.52470.log: -- replaced with /var/log/ceph/ceph-client.admin.39661.log.gz 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52470.log.gz 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45365.log 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53425.log.gz 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45365.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45365.log.gz 2024-09-17T17:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48398.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46139.log: 23.8% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48398.log: gzip -5 24.0% -- replaced with /var/log/ceph/ceph-client.admin.48398.log.gz 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55578.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45678.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52428.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55578.log.gz 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55410.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52428.log.gz 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55410.log.gzgzip 2024-09-17T17:32:34.727 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52260.log 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44076.log: /var/log/ceph/ceph-client.admin.43241.log: 24.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52932.log 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51882.log 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52260.log.gz 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52932.log.gz 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54066.log 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40551.log 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54066.log: /var/log/ceph/ceph-client.admin.51882.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54066.log.gz 2024-09-17T17:32:34.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41535.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40551.log: 24.0% -- replaced with /var/log/ceph/ceph-client.admin.40551.log.gz 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gz 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51709.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54948.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51709.log: /var/log/ceph/ceph-client.admin.41535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51709.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41535.log.gz 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38360.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54948.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54906.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54948.log.gz 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55956.log 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38360.log: /var/log/ceph/ceph-client.admin.54906.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38360.log.gz 2024-09-17T17:32:34.729 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54906.log.gz 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54654.log 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55956.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55200.log 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55956.log.gz 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55536.log 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54654.log: /var/log/ceph/ceph-client.admin.55200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55200.log.gz 26.8% 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54654.log.gz 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38788.log 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52890.log 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38788.log: /var/log/ceph/ceph-client.admin.55536.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55536.log.gz 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38788.log.gz 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56124.log 2024-09-17T17:32:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52890.log: /var/log/ceph/ceph-client.admin.56124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52890.log.gz 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56124.log.gz 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38330.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38330.log.gzgzip 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39787.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52554.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39787.log: /var/log/ceph/ceph-client.admin.45040.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39787.log.gz 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41245.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53462.log 2024-09-17T17:32:34.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41245.log: /var/log/ceph/ceph-client.admin.52554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52554.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41245.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45823.log 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53462.log: /var/log/ceph/ceph-client.admin.43901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53462.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47761.log 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45823.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47761.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47761.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52638.log 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-09-17T17:32:34.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54696.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52638.log.gz 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54696.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37939.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54696.log.gz 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52050.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38908.log: /var/log/ceph/ceph-client.admin.37939.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37939.log.gz 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54102.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52050.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37897.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42347.log 2024-09-17T17:32:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54102.log: /var/log/ceph/ceph-client.admin.37897.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54102.log.gz 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37897.log.gz 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55788.log 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42347.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55284.log 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39488.log 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55788.log: /var/log/ceph/ceph-client.admin.55284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55788.log.gz 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55284.log.gz 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46696.log 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39488.log.gz 2024-09-17T17:32:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46696.log.gz 2024-09-17T17:32:34.735 INFO:teuthology.orchestra.run.smithi026.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T17:32:34.735 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T17:32:34.754 INFO:teuthology.orchestra.run.smithi007.stderr: 94.4% -- replaced with /var/log/ceph/objectstore_tool.49134.log.gz 2024-09-17T17:32:35.005 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T17:32:35.011 INFO:teuthology.orchestra.run.smithi027.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T17:32:35.115 INFO:teuthology.orchestra.run.smithi027.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-09-17T17:32:35.148 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-09-17T17:32:35.163 INFO:teuthology.orchestra.run.smithi102.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T17:32:35.188 INFO:teuthology.orchestra.run.smithi007.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T17:32:35.244 INFO:teuthology.orchestra.run.smithi027.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-09-17T17:32:35.428 INFO:teuthology.orchestra.run.smithi026.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-09-17T17:32:35.485 INFO:teuthology.orchestra.run.smithi026.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-17T17:32:35.543 INFO:teuthology.orchestra.run.smithi102.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T17:32:35.559 INFO:teuthology.orchestra.run.smithi026.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T17:32:35.572 INFO:teuthology.orchestra.run.smithi007.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-17T17:32:35.573 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T17:32:35.704 INFO:teuthology.orchestra.run.smithi102.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-09-17T17:32:35.706 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-17T17:32:35.706 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m1.153s 2024-09-17T17:32:35.706 INFO:teuthology.orchestra.run.smithi102.stderr:user 0m3.234s 2024-09-17T17:32:35.706 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m0.138s 2024-09-17T17:32:35.714 INFO:teuthology.orchestra.run.smithi027.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T17:32:35.715 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-17T17:32:35.715 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m1.166s 2024-09-17T17:32:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m2.899s 2024-09-17T17:32:35.716 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.102s 2024-09-17T17:32:35.727 INFO:teuthology.orchestra.run.smithi026.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T17:32:35.728 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-17T17:32:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m1.179s 2024-09-17T17:32:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m3.956s 2024-09-17T17:32:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.162s 2024-09-17T17:32:35.776 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-09-17T17:32:35.778 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T17:32:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m1.230s 2024-09-17T17:32:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m4.240s 2024-09-17T17:32:35.778 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.302s 2024-09-17T17:32:35.778 INFO:tasks.ceph:Archiving logs... 2024-09-17T17:32:35.778 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi007/log 2024-09-17T17:32:35.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T17:32:36.662 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi026/log 2024-09-17T17:32:36.662 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T17:32:37.293 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi027/log 2024-09-17T17:32:37.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T17:32:37.777 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi102/log 2024-09-17T17:32:37.778 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T17:32:38.336 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T17:32:38.345 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T17:32:38.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T17:32:38.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T17:32:38.350 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T17:32:38.353 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T17:32:38.474 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T17:32:38.482 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T17:32:38.483 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:32:38.486 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:32:38.488 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:32:38.491 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T17:32:38.501 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:32:38.504 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:32:38.506 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:32:38.507 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:32:38.507 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-17T17:32:38.507 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +204us[ +203us] +/- 47ms 2024-09-17T17:32:38.507 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 +256us[ +256us] +/- 49ms 2024-09-17T17:32:38.507 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -856us[ -858us] +/- 88ms 2024-09-17T17:32:38.507 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:32:38.508 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-17T17:32:38.509 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:32:38.509 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-09-17T17:32:38.509 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +220us[ +226us] +/- 47ms 2024-09-17T17:32:38.509 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +243us[ +243us] +/- 49ms 2024-09-17T17:32:38.510 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -839us[ -839us] +/- 88ms 2024-09-17T17:32:38.510 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:32:38.511 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:32:38.511 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-17T17:32:38.511 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +223us[ +231us] +/- 47ms 2024-09-17T17:32:38.512 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +249us[ +249us] +/- 49ms 2024-09-17T17:32:38.512 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -849us[ -849us] +/- 88ms 2024-09-17T17:32:38.512 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:32:38.513 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T17:32:38.513 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-17T17:32:38.514 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +214us[ +218us] +/- 47ms 2024-09-17T17:32:38.514 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +250us[ +255us] +/- 49ms 2024-09-17T17:32:38.514 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -839us[ -835us] +/- 88ms 2024-09-17T17:32:38.514 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T17:32:38.514 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T17:32:38.523 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T17:32:38.523 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T17:32:38.546 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T17:32:38.630 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T17:32:38.703 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T17:32:38.777 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T17:32:38.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:32:38.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:38.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:32:38.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:38.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:32:38.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:38.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T17:32:38.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:38.976 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T17:32:38.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T17:32:39.004 INFO:teuthology.task.internal:Duration was 673.372827 seconds 2024-09-17T17:32:39.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T17:32:39.038 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T17:32:39.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T17:32:39.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T17:32:39.043 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T17:32:39.046 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T17:32:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:32:39.087 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:32:39.087 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:32:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T17:32:39.594 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T17:32:39.595 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T17:32:39.595 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T17:32:39.621 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-09-17T17:32:39.621 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T17:32:39.655 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-17T17:32:39.656 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T17:32:39.680 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-17T17:32:39.680 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T17:32:39.706 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T17:32:39.706 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T17:32:39.709 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T17:32:39.711 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T17:32:39.723 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T17:32:39.774 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T17:32:39.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T17:32:39.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T17:32:39.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T17:32:39.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T17:32:39.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T17:32:39.929 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T17:32:39.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T17:32:39.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T17:32:39.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T17:32:40.008 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T17:32:40.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T17:32:40.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.049 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.051 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.068 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-17T17:32:40.071 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-09-17T17:32:40.073 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-17T17:32:40.097 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-17T17:32:40.121 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:40.158 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:40.174 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:40.189 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T17:32:40.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T17:32:40.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T17:32:40.213 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T17:32:40.214 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi007 2024-09-17T17:32:40.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T17:32:40.249 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi026 2024-09-17T17:32:40.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T17:32:40.283 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi027 2024-09-17T17:32:40.283 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T17:32:40.317 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909156/remote/smithi102 2024-09-17T17:32:40.317 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T17:32:40.350 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T17:32:40.350 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T17:32:40.352 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T17:32:40.354 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T17:32:40.357 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T17:32:40.400 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T17:32:40.413 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T17:32:40.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T17:32:40.430 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T17:32:40.430 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T17:32:40.432 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T17:32:40.435 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T17:32:40.437 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T17:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 17:32 /home/ubuntu/cephtest 2024-09-17T17:32:40.448 INFO:teuthology.orchestra.run.smithi026.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 17:32 /home/ubuntu/cephtest 2024-09-17T17:32:40.450 INFO:teuthology.orchestra.run.smithi027.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 17:32 /home/ubuntu/cephtest 2024-09-17T17:32:40.455 INFO:teuthology.orchestra.run.smithi102.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 17:32 /home/ubuntu/cephtest 2024-09-17T17:32:40.456 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T17:32:40.465 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T17:32:40.533 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-hybrid} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 673.3728265762329 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-17T17:32:40.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T17:32:40.629 INFO:teuthology.run:pass